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ABSTRACT 


The  initial  phase  of  our  synthetic  discriminant  function  optical  pattern 
recognition  research  has  been  completed.  This  hyperspace  description  allows 
the  use  of  off-line  synthesis  of  a  filter  function  that  provides  both  intra¬ 
class  recognition  (of  a  key  object  independent  of  geometrical  distortions)  and 
inter-class  discrimination  (of  a  key  object  from  candidate  false  targets). 

A  new  theoretical  model  for  multi-sensor  data,  a  description  of  a  general 
multi-sensor  processor  and  a  statistical  analysis  of  both  have  also  been 
completed.  Initial  experimental  correlations  were  obtained  and  found  to  be 
in  agreement  with  our  model  and  theoretical  analysis.  A  new  parameter,  the 
correlation  measure,  was  found  to  be  of  use  in  determining  the  multi-sensor 
nature  of  input  imagery  and  a  |GRAD|  digital  preprocessing  operator  was  found 
to  be  quite  adequate  for  the  cases  considered. 

Our  initial  optical  correlations  on  the  autonomous  terminal  homing  data 
base  were  also  completed.  In  this  research,  aperture  correlation  effects  were 
overcome  by  a  new  technique,  optical  and  digital  correlations  on  the  same  data 
base  were  obtained  and  compared  (with  a  high  pass  filter  model  for  the  optical 
matched  filter  found  to  be  adequate  and  appropriate  for  many  applications),  and 
sensed/synthetic  optical  image  correlations  were  obtained  (with  the  help  of 
photoreduced  imagery  and  advanced  preprocessing  operations). 
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1 .  INTRODUCTION 


During  the  past  year,  our  research  in  optical  data  processing  for  missile 
guidance  has  addressed  various  new  algorithms,  system  architectures,  component 
tests,  and  analyses  of  various  image  data  bases.  As  in  the  past  years,  we  have 
been  quite  faithful  in  reporting  our  AFOSR  sponsored  research  in  various  journal 
and  conference  publications.  Copies  of  the  more  relevant  of  these  papers  are 
thus  included  as  the  chapters  of  this  report  to  provide  concise  documentation 
of  our  work. 

In  Section  2,  we  provide  a  summary  and  overview  of  our  research  progress 
made  in  the  past  year.  Details  on  the  more  salient  topics  are  provided  in 
Sections  3-7.  In  Section  9,  we  enumerate  our  AFOSR  sponsored  publications 
and  the  presentations  given  on  this  research  in  the  past  year. 
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2.  SUMMARY  AND  OVERVIEW 

2.1  AIRBORNE  OPTICAL  CORRELATOR  (Section  3) 

Since  the  intended  application  for  most  of  our  pattern  recognition  tech¬ 
niques  is  missile  guidance,  attention  to  the  components  for  use  in  such  a 
system  is  necessary  together  with  a  rugged  and  stable  system  architecture.  In 
[1],  we  first  addressed  this  subject.  In  [2]  and  [3],  we  provided  additional 
details  of  this  work.  In  [4]  (Section  3),  we  summarize  these  results  with 
attention  to:  the  use  of  laser  diode  sources  (rather  than  the  larger  laser 
sources  used  in  the  conventional  laboratory  optical  system),  holographic  optical 
elements  (rather  than  the  larger  and  heavier  conventional  glass  lenses),  plus 
a  new  architecture  (the  lensless  matched  spatial  filter)  with  greatly  improved 
stability  and  ruggedness. 

The  basic  system  architecture  involves  imaging  the  laser  diode  source  onto 
a  lensless  matched  filter  with  the  input  placed  behind  the  lens.  The  lensless 
matched  filter  is  a  combined  matched  filter  and  holographic  optical  element 
(formed  with  a  converging  reference  beam,  rather  than  the  conventional  plane 
wave  reference  beam  normally  used).  When  part  of  the  input  scene  contains  the 
reference  object,  a  focused  beam  emerges  from  the  matched  spatial  filter  and 
self  focuses  on  the  output  correlation  plane.  Since  the  second  Fourier  transform 
lens  and  the  matched  spatial  filter  are  recorded  on  the  same  plate,  the  position¬ 
ing  tolerances  between  these  two  components  are  inherently  satisfied.  This  greatly 
simplifies  the  stability  and  ruggedness  of  the  resultant  system.  The  matched 
spatial  filter  is  formed  in  visible  light  at  one  wavelength  X^  (for  which  spec¬ 
troscopic  grade  film  of  adequate  sensitivity  is  available),  whereas  correlation 
is  performed  with  a  laser  diode  source  at  a  second  wavelength  X^.  The  architecture 
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of  the  scaling  correlator  system  used  is  adjusted  to  correct  for  different 
recording  and  reading  wavelengths. 

Our  present  work  (Section  3)  has  included  an  analysis  of  the  spatial  and 
temporal  coherence  of  the  laser  diode  source  and  the  shift  invariance  and 
aberrations  of  the  holographic  optical  elements  used.  Future  work  in  this 
area  is  anticipated  and  will  be  reported  upon  in  future  annual  reports.  The 
general  philosophy  used  in  our  lensless  matched  spatial  filter  correlator  will 
be  pursued  in  our  future  work  (i.e.  alterations  of  the  standard  optical  proces¬ 
sor  architectures,  when  non-convent ional  components,  such  as  holographic  elements, 
or  laser  diodes,  are  employed). 

2.2  SPATIAL  LIGHT  MODULATORS 


A  vital  feature  of  most  optical  data  processing  systems  is  the  real  time 
and  reuseable  2-D  spatial  light  modulator  necessary  to  input  data  to  the  optical 
processor.  During  the  past  year,  we  have  performed  a  test  and  evaluation  of 
five  Soviet  Trom  and  Priz  light  modulators.  The  results  of  this  phase  of  our 
research  were  most  attractive  and  will  be  presented  later  when  available  [5-8]. 

2.3  OPTICAL  EXPERIMENTS  FOR  AUTONOMOUS  TERMINAL  HOMING  (Section  4) 

The  major  image  data  base  presently  available  is  the  autonomous  terminal 
homing  DARPA  set.  We  have  completed  our  initial  pattern  recognition  analysis 
on  this  data  base  (Section  4)  [9].  The  optical  system  architecture  used  was  a 
weighted  matched  spatial  filter  correlator.  The  low  resolution  and  modulation 
of  this  data  base  introduced  a  new  problem  (the  dominance  of  aperture  correla¬ 
tions  over  image  correlations),  which  we  overcame  by  the  use  of  different  aper- 
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tures  during  synthesis  of  the  matched  spatial  filter  and  during  correlation 
with  the  on-line  sensed  image.  In  this  study,  we  also  provided  the  first 
direct  comparison  of  optically  and  digitally  correlated  data  (with  various 
digital  preprocessing  operations  used).  From  these  tests,  we  found  that  a 
highpass  filter  model  was  quite  appropriate  for  an  optical  weighted  matched 
spatial  filter  correlator.  The  experiments  were  performed  on  multi-sensor 
data  and  from  comparative  results  obtained  with  different  preprocessing  opera¬ 
tors,  we  found  that  a  simple  highpass  filtering  preprocessing  operator  was 
often  adequate.  For  more  true  multi-sensor  data  with  a  small  correlation 
measure  (see  Section  5),  the  |GRAD|  operator  (obtained  by  digital  preprocessing) 
was  preferable. 

The  final  feature  of  this  research  was  our  first  successful  optical  cor¬ 
relations  on  sensed  input  imagery  and  synthetic  reference  data.  As  our  work 
showed,  such  correlations  were  possible  only  when  aperture  effects  were  removed, 
when  the  input  imagery  was  photoreduced,  and  when  advanced  preprocessing  operators 
(e.g.  gray  tone  modification  and  small  surface  lysing)  were  applied  to  the  data 
bases . 

Analysis  of  this  data  base  and  more  attention  to  different  preprocessing 
operators  is  required  and  will  be  the  subject  of  future  work  by  us. 

2.4  MULTISENSOR  PATTERN  RECOGNITION  (Section  5) 

A  major  class  of  imagery  on  which  considerable  pattern  recognition  research 
has  been  performed  is  multi-sensor  data.  In  [10],  we  described  a  new  statistical 
model  for  multi-sensor  imagery  and  a  general  multi-sensor  processor.  We  performed  a 


a  statistical  analysis  of  our  general  processor  using  our  image  model.  In  [11] 
(Section  5),  we  describe  our  recent  results  in  which  experimental  verification 
of  our  model  and  processor  were  obtained.  These  tests  verified  the  use  of 
our  new  parameter  (the  correlation  measure)  as  a  measure  of  whether  the  image 
pairs  were  unisensor  or  multi-sensor  imagery.  These  tests  also  concentrated 
upon  the  use  of  and  need  for  various  preprocessing  operators  in  the  correlation 
(for  acquisition)  of  multi-sensor  data. 

For  image  pairs  with  high  correlation  measures,  the  data  was  found  to  be 
uni-sensor  and  a  highpass  filter  preprocessing  operator  was  adequate.  For  data 
with  low  (<  0.5)  correlation  measures,  a  nonlinear  magnitude  operator  provided 
improved  results.  We  also  found  the  |GRAD|  operator  to  well  approximate 
whitening,  edge  enhancement  and  polarity  equilization.  The  use  of  histogram 
reshaping  as  a  preprocessing  operator  was  found  to  provide  a  6-32%  improvement 
for  different  multi-sensor  image  sets.  As  the  number  of  objects  in  the  image 
was  increased,  we  found  the  data  to  more  closely  approach  multi-sensor  imagery. 
Our  theoretical  threshold  values  were  also  found  to  be  verified  by  experiment. 
During  the  next  year,  we  will  more  completely  document  and  analyze  these  initial 
results . 

2.5  INTRA-CLASS  PATTERN  RECOGNITION  USING  SYNTHETIC  DISCRIMINANT  FUNCTIONS 

(Section  6) 

Maintaining  pattern  recognition  in  the  face  of  geometrical  distortions 
between  the  input  and  reference  data  (especially  3-D  aspect  angle  differences) 
is  perhaps  one  of  the  most  difficult  data  processing  problems  in  existence. 

To  address  this  problem,  we  suggested  [12]  the  use  of  a  hyperspace  formulation 
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of  Che  data  for  the  problem  in  which  each  input  image  is  represented  by  a  vector 
point  in  a  hyperspace.  Our  initial  technique  has  been  refined  in  [13]  (Sec¬ 
tion  6)  to  include  a  new  maximum  common  information  concept  and  the  use  of 
other  basis  function  synthesis  techniques.  This  approach  allows  a  synthetic 
discriminant  function  (defined  by  a  linear  combination  of  the  image  training 
set)  to  be  produced  off-line  and  to  be  subsequently  used  in  the  future  recog¬ 
nition  of  the  same  target  independent  of  its  geometrical  aspect. 

Although  our  initial  results  are  most  promising,  much  additional  work  is 
11  *  necessary  on  this  concept.  We  intend  to  extensively  pursue  this  concept  during 

the  coming  years. 

2.6  INTER-CLASS  DISCRIMINATION  USING  SYNTHETIC  DISCRIMINANT  FUNCTIONS 
(Section  7) 

As  an  initial  extension  of  our  synthetic  discriminant  function  research 
(Section  6),  we  considered  the  case  of  discrimination  in  the  two-class  pattern 
recognition  problem  and  suggested  several  new  techniques  and  provided  initial 
experimental  verification  of  them  [14]  (Section  7).  These  concepts  included: 
maximum  common  information  filters,  multi-channel  filters,  non-unitary  trans¬ 
formations,  and  decorrelation  transformations.  Extensive  additional  research 
on  this  aspect  of  the  synthetic  discriminant  function  concept  with  attention 
to  multi-class  pattern  recognition  problems  and  statistical  filters  is  the  sub¬ 
ject  of  current  research  that  will  be  reported  upon  in  future  annual  reports. 

2.7  ITERATIVE  OPTICAL  PROCESSOR 

Due  to  cancellation  of  other  support  for  our  iterative  optical  processor 
[15]  research,  interim  temporary  funding  of  this  project  by  AFOSR  has  enabled 
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us  to  complete  a  Ph.D  thesis  in  this  area,  to  generalize  on  the  different  ap¬ 
plications  possible  for  this  optical  system  architecture  [16],  and  to  document 
our  work  in  this  area  [17-19],  We  will  detail  and  highlight  this  phase  of 
our  research  in  our  next  annual  report  (after  publications  on  the  subject  are 
finalized)  . 

2.8  FUTURE  RESEARCH  DIRECTION 

Our  future  research  is  expected  to  follow  the  major  lines  addressed  in 
the  past  two  years  (real-time  devices,  optical  system  components  and  airborne 
processors)  with  major  attention  given  to  new  pattern  recognition  architectures 
and  algorithms  (especially  hybrid  optical/digital  systems).  As  can  be  seen,  the 
flavoring  of  our  recent  and  future  research  is  being  directed  toward  more  use 
of  statistical  analysis  and  digital  simulation  of  optical  systems.  We  will  also 
continue  to  apply  our  new  algorithms,  architectures  and  concepts  to  new  missile 
guidance  image  data  bases.  Several  general  pattern  recognition  reviews  were 
published  during  the  past  year  under  AFOSR  support  and  may  be  useful  for  a  general 
review  of  this  area  [20-23], 
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STABLE  AND  COMPACT  AIRBORNE  OPTICAL 
PATTERN  RECOGNITION  SYSTEM  CONCEPT 


A  LASER  DIODE/LENSLESS  MSF  OPTICAL  PATTERN  RECOGNITION  SYSTEM 

by 


D.  Casasent,  M.  Shen*,  F.  Calrol,  T.  Luu  and  B.  Feng* 
Carnegie-Mellon  University 
Department  of  Electrical  Engineering 
Pittsburgh,  Pennsylvania  15213 


ABSTRACT 


A  compact  and  rugged  optical  correlator  is  described  that  uses  a  laser  diode  source  and  holo¬ 
graphic  optical  elements.  Use  of  a  lensless  matched  spatial  filter,  and  corrections  for  wavelength 
scaling  and  aberration  errors  plus  an  analysis  of  the  system's  shift  Invariance  are  Included. 

1.  INTRODUCTION 


Optical  Pattern  recognition  (OPR)  by  matched  spatial  filter  (MSF)  correlation  is  well-known  II). 
The  advent  of  holographic  optical  elements  (HOEs)  (2-3),  laser  diode  sources  and  new  architectures  has 
made  rugged  and  compact  OPR  systems  roost  attractive  [4-7)  for  commercial  and  airborne  applications. 

In  this  paper  we  describe  the  first  system  [7]  to  employ  both  solid-state  sources  and  HOEs.  A  lens¬ 
less  MSF  in  which  the  MSF  and  second  Fourier  transform  (FT)  lens  are  combined  on  one  plate  1 8-9]  was 
used  to  avoid  critical  alignment  of  these  two  components.  The  design,  experimental  verification  and 
shift  Invariance  of  this  system  are  discussed  in  Sects  2.  When  a*  laser  diode  source  is  used,  the  MSF 
is  recorded  at  one  wavelength  and  correlation  is  performed  at  a  second  wavelength.  A  new  scaling  cor¬ 
relator  topology  using  an  imaging  lens  was  employed  to  correct  for  wavelength  changes.  The  design, 
fabrication,  aberration  analysis  and  experimental  demonstration  of  the  system  are  presented  in  Sect.  3. 

2.  LENSLESS  MSF 

2.1  THEORY 


The  system  used  to  record  a  lensless  MSF  in  shown  in  Fig.  1.  It  It  similar  to  the  conventional 
technique  II)  except  that  a  converging  reference  beam  is  used  rather  than  a  plane  wave.  For  simplic¬ 
ity  of  notation,  only  a  1-D  analysis  Is  included  with  no  loss  of  generality.  During  MSF  synthesis, 
h(x)  is  placed  at  Pj  and  its  FT  incident  on  Pj  is  us(x2)  *  H(x2/7fi_).  The  spatial  coordinates  of  Pj, 
P2,  and  P3  are  denoted  by  xi,  X2,  and  X3  respectively;  1  is  the  wavelength  of  the  incident  light  and 
fj,  is  the  focal  length  of  lens  L3.  The  converging  reference  beam  at  P2  is  described  by 

ur(x2)  "  exp[-Jk(x0-X2)2/2z) ,  (1) 

where  k  •  2v/\,  xq  is  the  distance  from  the  axis  in  P3  at  which  the  converging  beam  focuses,  and  z  is 
the  orthogonal  distance  from  P2  to  P3.  The  pattern  recorded  at  P2  is  |us+ur|’.  We  assume  that  the 
transmittance  of  P2  after  development  is  equal  to  the  incident  exposure  and  thus  the  term  of  .nterest 
in  this  MSF  at  P2  is 

ut(x2)  *  H*(x2/7fL)exp[-jk(x0-x2)2/2z) ,  (2) 

where  the  constant  phase  factor  describing  its  propagation  direction  is  omitted  [or  simplicity. 

If  P2  is  re-illuminated  with  ug,  a  point  of  light  appears  at  X3  «  x^ .  Our  major  concern  is  that 
this  occurs  without  the  need  for  a  second  FT  lens  between  P2  and  P3.  Thus  no  critical  alignment  of 
chese  components  is  needed  in  this  lensless  MSF  system.  If  h  is  shifted  in  Pj  during  reconstruction, 
the  locatlonof  the  point  of  light  in  P3  should  shift  from  its  original  xg  location  by  an  amount  pro¬ 
portional  to  the  shift  of  h  in  Pi.  This  shift  invariance  is  required  in  pattern  recognition. 

To  see  the  use  of  such  a  system  for  pattern  recognition,  we  place  f(xj)  *  h(xi+’ )  +  g(xj)  at  Pj. 
This  function  contains  the  reference  h(xj)  displaced  from  the  origin  by  an  amount  r  and  gfxj)  denotes 
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Che  rest  of  the  input.  We  assume  g0h  -  0  and  describe  the  light  distribution  incident  on  P2  for  the 
term  of  interest  by 

UjCxj)  -  H(x2/XfL)exp(jkx2c/fL) .  (3) 

Leaving  P2>  we  have  (suppressing  constant  phase  terms  for  simplicity) 

u2(x2)u£(x2)  «  H(x2/XfL)H*(x2/XfL)exp(-Jkx2/2z)exp[jkx2(c/fL+x0/z)].  (4) 

At  P3  we  find  the  FT  of  (4)  or  (omitting  a  quadratic  phase  factor,  that  is  negligible  when  intensity 
is  detected  at  P3) 

u3(x3)  -  h(x3fL/z)0h(x3fL/z)*6(x3+xQ+Ez/fL).  '  (5) 

Inspection  of  (5)  shows  that  this  P3  pattern  is  the  correlation  of  h  displaced  from  the  reference 
point  xo  by  an  amount  zc/fL  proportional  to  the  displacement  e  of  h  from  the  center  of  Pj. 

Many  authors  have  proposed  (and  some  have  used)  convergent  reference  beams  for  diverse  purposes 
(8,  9,  11-13].  In  our  work,  we  are  concerned  with  the  aberration  effects  of  the  HOE  on  the  system's 
shift  invariance  as  well  as  the  use  of  the  system  for  pattern  recognition  [14]. 

2 . 2  EXPERIMENTS 


The  system  of  Fig.  1  was  assembled  using:  X  *  633  nm,  f l  ■  500  mm,  0  •  11.3°  and  xq  «  78  mm. 
The  input  function  f  used  is  shown  in  Fig.  2a.  It  contains  four  occurrences  of  the  key  word  "PRO¬ 
FESSOR",  which  we  select  as  our  reference  function  h  (Fig.  2b).  A  lensless  MSF  of  this  function  h 
was  produced  at  P2,  f  was  placed  at  Pi  and  the  output  correlation  plane  pattern  in  the  vicinity  of 
(X3,  y3)  ”  (xo,0)  in  P3  appeared  as  shown  in  Fig.  2c  on  an  isometric  display  [14],  The  locations  of 
the  four  correlation  peaks  are  seen  to  correspond  to  the  four  positions  of  the  reference  word  in  the 
full  input  image. 

2.3  ANALYSIS 


A  feature  of  this  and  any  MSF  system  for  OPR  is  proper  selection  of  the  beam  balance  ratio 
K  «  |ur/us'|2  used  during  MSF  synthesis  [10].  Since  u8  varies  spatially,  so  does  K.  Since  the  modu¬ 
lation  of  the  MSF  is  a  maximum  when  K  »  1,  we  can  select  the  spatial  frequency  band  in  which  to  set 
K  »  1  and  hence  enhance  certain  spatial  frequencies  in  the  input  data  during  MSF  synthesis.  We  refer 
to  this  as  weighted  MSF  synthesis  [10].  For  the  input  text  data  of  Fig.  2a,  the  FT  of  the  word  "PRO¬ 
FESSOR"  was  found  to  exhibit  peaks  at  spatial  frequencies  corresponding  to  the  reciprocal  of  the: 
stroke  width,  spacing  between  letters,  the  width  of  a  letter  and  half  the  height  of  a  letter.  The  FT 
information  in  the  word  is  present  as  modulation  about  these  spatial  frequencies  [15).  The  latter  two 
spatial  frequencies  (letter  width  aid  half-letter  height)  coincide  at  3.7  cy/mm,  where  a  bright  FT 
plane  spot  is  observed.  We  adjusted  K  to  be  one  at  this  spatial  frequency  during  synthesis  of  the  MSF 
and  thus  maximized  the  correlator's  performance.  Such  an  MSF  acts  as  a  bandpass  filter  centered  at 
3.7  cy/mm.  Because  of  this,  the  HOE  need  only  operate  over  a  band  of  spatial  frequencies  and  thus  its 
performance  is  less  demanding  than  that  of  the  normal  FT  or  HOE  lens. 

We  now  consider  the  effect  of  weighted  MSF  synthesis  on  HOE  aberrations  and  the  system's  shift 
invariance.  Particular  attention  is  given  to  astigmatic  aberrations  [9,  16].  Including  third  order 
aberrations,  denoting  the  field  angle  of  the  key  object  by  tan  8  -  c/fi_  an<*  assuming  tan  r  <<  2xo/Ro> 
where  Rg  is  the  distance  from  the  center  of  P2  to  X3  *  xg,  we  find  the  optical  path  difference  for  the 
system  to  be  Ara  -  XT^Xgftan  6)/Rg^  and  (tan  B)max  “  D/2f,,  where  D  is  the  input  aperture.  If  the 
maximum  input  spatial  frequency  is  fn,  the  maximum  optical  path  difference  error  is 

Ar  -  f  2X2Df,  sine/2R„,  (6) 

a  max  m  L  0 

where  ain0  ■  xg/Rg.  In  the  design  of  our  experimental  system,  we  restricted  fm  and  D,  minimized  • 
and  fj_,  and  increased  Rg  to  reduce  (6).  From  [16],  the  astigmatic  phase  aberration  Ca  expression  was 
found.  A  quadratic  phase  error  model  was  assumed.  From  [17]  the  loss  p  in  the  correlation  peak 
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(7) 


intensity  Ip  was  found  to  be  related  to  the  standard  deviation  6^  of  over  the  full  aperture  by 


Inserting  system  values  into  (7),  a  plot  of  p  for  input  displacements  Ax,  from  +10  mm  to  -10  an 
was  made.  From  this  theoretical  curve,  a  maximum  Ip  variation  of  only  20Z  was  predicted  for  +10  aa 
displacements  of  h(x)  across  the  full  input  aperture.  The  nearly  uniform  values  for  the  four  correla¬ 
tion  peaks  In  Fig.  2c  verifies  the  excellent  ahlft-lnvarlance  of  this  system  design.  Detailed  exper¬ 
imental  measurements  of  I  were  made  for  +10  am  input  displacements  and  a  maximum  Ip  loss  of  only  10Z 
(or  half  the  theoretically  predicted  value)  was  found.  The  superiority  of  the  experimental  data  to 
the  theoretical  predictions  is  attributed  to  the  bandpass  effects  of  the  weighted  MSF.  Recall  that 
the  above  theory  assumed  that  the  HOE  aberrations  existed  over  the  entire  spatial  frequency  aperture 
of  the  HOE.  Thus  weighted  MSF  synthesis  [10]  reduces  HOE  aberration  effects  and  aakes  the  design  of 
a  shift-invariant  lensless  MSF  correlator  with  low  10Z  variations  In  Ip  realistic  [14], 


3.  LASER  DIODE  CORRELATOR 


When  a  laser  diode  at  X2  1®  used  as  the  light  source  during  correlation.  Its  typical  >2  value  is 
not  In  the  wavelength  range  of  MSF  media.  Thus  the  lensless  HOE/MSF  at  P2  must  be  synthesized  off¬ 
line  at  X^  and  correlation  performed  at  the  different  X2  of  the  laser  diode.  This  u  -  X2/Xj  wave¬ 
length  change  is  equivalent  to  a  scale  error  In  the  Imagery.  In  this  section,  we  discuss  this  aspect 
of  laser  diode  correlators  and  other  laser  diode  parameters  and  their  effects  on  the  system's  design 
and  performance. 

3.1  WAVELENGTH-SCALING  CORRELATOR 

The  laser  diode  correlator  used  Is  shown  schematically  in  Fig.  3.  During  MSF  synthesis,  Lj  Is 
illuminated  with  a  plane  wave  at  Xj  and  the  converging  reference  beam  Is  used.  With  P3  placed  behind 
L* ,  the  FT  of  the  Pj  data  is  scaled  [18]  by  fL/d.  By  varying  d,  the  scale  change  w  can  be  compensated. 
To  satisfy  the  focusing  conditions,  the  scale  change  must  be  kept  below  20Z  [10].  We  denote  the  dis¬ 
tances  d  and  e  and  the  angle  0  In  Fig.  3  during  MSF  synthesis  and  correlation  by  subscripts  1  and  2. 

To  compensate  for  the  scale  change  u  and  to  satisfy  focusing  conditions,  ye  must  satisfy  d2  *  dj/u, 
e2  “  ej/p  and  sinBj  *  psin02-  The  displacement  Ax 3  of  the  output  correlation  Is  related  to  the  dis¬ 
placement  Axj  of  the  Input  by  Lxj/Lx^  -  e2/d2  “  el'dl‘  By  choOBln8  **1  <<:  d2  Bnd  ^*3  ej  •  field 
angles  and  hence  distortion  effects  will  be  small  and  HOE  aberration  effects  will  be  minimized. 

Such  a  wavelength-scaled  correlator  was  assembled  and  tested  [19]  using  gas  lasers  (Xj  •  488  nm 
Argon  laser,  X2  -  633  nm  He-Ne  laser).  Less  than  10Z  variation  in  Ip  were  observed  for  +10  mm  input 
displacements.  A  system  with  HOE  elements  for  both  L}  and  L2  was  also  assembled  and  correlations  were 
successfully  performed  with  gas  laser  sources  [19].  Although. the  output  light  level  was  low  due  to 
the  low  diffraction  efficiency  of  the  HOE  elements,  acceptable  output  correlations  were  obtained. 

Better  results  are  possible  If  bleached  or  dlchromated  gelatin  elements  (2,  3]  with  higher  (80Z  vs  3Z) 
diffraction  efficiency  were  used. 

• 

3.2  ABERRATION  ANALYSIS 


For  the  laser  diode  correlator,  we  use  Xj  »  633  nm  and  X2  ■  794.5  nm  (Mitsubishi  ML-4001  laser 
diode).  The  astigmatic  aberration  equation  of  [16],  in  our  notation,  becomes 

Wa  “  (xm2/2A2)((lj3'lj),t02/el3  +  2g3VV*l2dl  +  <«> 

The  system  parameters  chosen  for  our  correlator  of  Fig.  3  were:  ej  “  dj  -  400  mm,  fj,i  “  760  mm, 
fB  *  10  cy/an,  x,,,  *  2.5  mm,  Axj  “  12.5  an  and  x-  ■  100  mm.  For  these  values,  we  find  W  *  0.44  '2 

and  Va  »  0.13  X2.  These  low  phase  errors  should  result  in  a  high  performance  correlator.  Recall 
that  weighted  MSF  synthesis  will  reduce  these  W(  effects  even  more. 
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3.3  LASER  DIODE  EFFECTS 


The  temporal  coherence  of  a  laser  diode  yarles  with  power  and  temperature  and  can  result  In  a 
source  width  AA2  and/or  a  wavelength  shift  AAj.  For  the  laser  diode  used,  “  0-2  nm  and  AA2  „  2nm. 

For  our  case  N  ^  A2/AA2  =  400.  We  expect  the  system  to  operate  well  for  higher  N  because  most 
of  the  contribution  to  the  correlation  Is  due  to  spatial  frequency  data  below  the  maximum  and  the 
above  result  thus  gives  a  quite  worst-case  limit  on  N.  The  spectral  spread  A>2  of  the  source  wil., 
produce  a  divergence  A02  In  the  wavefront  leaving  P2  during  correlation.  This  can  affect  the  shape 
and  1^  value  of  the  correlation.  To  reduce  this  effect,  we  require  [5,  7} 

AX 2 ><  *22/(2wxb  sin  6^.  (9) 

For  our  system,  sin  8j  »  0.25,  p  •  1.26  and  thus  satisfying  (9)  requires  AX2  .$  0.4  nm.  Since  the  AA2 
of  our  laser  diode  source  is  half  of  this  value,  we  expect  no  problems  from  this  error  source  (with 
our  selected  system  design).  Only  AXj  not  AA2  affects  the  shape  of  the  correlation  whereas  the  larger 
of  AA2  and  A>2  affects  Nt  the  space  bandwidth  of  the  Input  in  1-D. 

The  spatial  coherence  of  the  laser  diode  source  used  was  also  considered.  This  parameter  was 
obtained  from  fringe  visibility  measurements.  Over  a  25  m  Input  aperture,  V  was  a. minimum  of  0.75. 
From  [20],  we  define  V  as  the  ratio  of  coherent  Intensity  Ic  to  total  Intensity  It.  Assuming  uniform 
illumination  at  Pj  and  since  the  correlator's  output  Is  due  to. the  coherent  light  contribution,  we 
expect  the  correlation  peak  intensity  Ip  to  be  proportional  to  the  square  of  Ic.  We  thus  expect  cor¬ 
relation  Ip  and  SNR  to  be  decreased  by  a  factor  v2  due  to  the  limited  (V  d  1)  spatial  coherence  of  the 
laser  diode  used.  For  our  system,  the  source's  spatial  coherence  will  cause  only  a  2.5  dB  loss  in  SNR 
as  the  input  object  shifts  from  the  center  to  the  edge  of  the  Pj  aperture.  This  is  again  a  worst-case 
result,  for  reasons  similar  to  those  noted  earlier. 

Proper  colllmation  of  the  laser  diode  source  should  also  be  Insured.  For  2u  “  10*  aperture  angle 
of  the  laser  diode  (see  Fig.  7)  and  its  wavelength,  the  source  can  be  considered  to  be  a  point.  We 
can  thus  use  a  simple  imaging  optical  lens  for  with  quite  modest  requirements  (a  cemented  doublet 
suffices).  A  trade  off  exists  between  uniformity  of  the  Illumination  at  Pi  (by  increasing  the  dis¬ 
tance  from  the  laser  diode  to  Lj )  and  less  light  power  in  the  Pi  aperture  (shorter  distance  fromtfie 
source  to  Pi).  For  the  system  design  used,  Li  Intercepted  a  7*  cone  angle.  For  this  case,  one-sixth 
of  the  source  light  is  useable  and  the  Pi  Illumination  has  a  30Z  Intensity  taper  (thus  a  3  dB  varia¬ 
tion  in  I  and  SNR  for  input  displacements  over  the  full  ?i  aperture  Is  expected,  again  in  worst- 
case).  These  various  source  coherence,  lens  aberration,  and  uniform  illumination  effects  can  add 
in  worst-case  to  produce  Ip  and  SNR  variations  of  6  dB  maximum. 

3.4  EXPERIMENTS 


All  the  above  analysis  Issues  were  Incorporated  into  the  design  of  an  experimental  system  In  the 
topology  of  Fig.  3.  In  Fig.  4  we  show  the  input,  reference  and  output  correlation  plane  pattern  ob¬ 
tained  with  the  MSF  formed  at  Aj  «  633  nm  and  correlation  performed  with  t  laser  diode  at  X2  •  795.4  nm. 
The  input  transparency  used  was  a  radar  image  with  a  24  mm  x  30  ran  format.  The  SNR  of  the  correlation 
is  seen  to  be  quite  good  even  though  the  reference  was  located  at  the  edge  of  the  input  scene. 

4.  SUMMARY  AND  CONCLUSIONS 

A  low  size  and  weight  optical  correlator  suitable  for  airborne  applications  has  been  described, 
designed,  analyzed,  fabricated,  tested  and  successfully  demonstrated.  The  system  employs  holographic 
optical  elements  in  a  lensless  MSF  architecture  with  improved  system  alignment  and  stability  features. 
The  light  source  used  during  correlation  was  a  laser  diode.  It  was  thus  necessary  to  record  the 
matched  spatial  filter  at  one  wavelength  and  perform  the  correlation  at  the  different  wavelength  of 
the  laser  diode.  A  scaling  correlator  architecture  was  used  to  accomodate  the  wavelength  change  and 
a  system  design  requiring  only  a  simple  imaging  input  lens  to  focus  the  laser  diode  onto  the  filter 
was  employed.  A  complete  system  design  and  analysis  including  aberration  effects  and  with  attention 
to  the  system's  shift-invariance  was  performed.  The  experimental  results  obtained  verified  the  use¬ 
fulness  of  this  system. 
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Figure  1  Schematic  diagram  of  a  lensless  MSF  optical  correlator  system  (14). 
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Optical  pattern  recognition  performed  on  Che  system  of  Fig.  O  (a)  input,  (h)  referent- 
(c)  output  correlation  plane  pattern  for  an  MSF  of  the  key  word  1R0FF.SSCK  1 
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Figure  3  Schematic  diagram  of  a  laser  diode/lensless  MSF  correlator  (7). 
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ABSTRACT 

Three  aspects  of  image  quality  and  their  effects  in  an  optical  matched  spatial  filter 
correlator  are  described.  These  concern  operation  on:  digitized  imagery,  data  with  low  mod¬ 
ulation  and  low  space  bandwidth  product,  and  synthetic  reference  imagery.  To  address  these 
practical  problems,  we  employ:  spatial  filtering,  edge  enhancement,  use  of  different  aper¬ 
tures,  photoreduced  imagery,  and  digital  preprocessed  data.  The  first  experimental  data  on 
optical  matched  spatial  filter  correlations  with  synthetic  reference  imagery  and  the  first 
comparative  data  on  digitally  and  optically  processed  multisensor  image  correlations  are 
included.  From  these  experiments,  we  find  an  optical  weighted  matched  spatial  filter  corre¬ 
lator  to  be  adequate  for  most  multisensor  data  and  that  advanced  digital  preprocessing  opera¬ 
tors  are  necessary  when  presently  available  synthetic  reference  imagery  is  used. 

1.  INTRODUCTION 

The  scaling  correlator  version  of  the  optical  frequency  plane  correlator  [1]  (Figure  1) 
using  weighted  matched  spatial  filter  (MSF)  synthesis  [2)  is  perhaps  the  most  widely  used 
optical  pattern  recognition  system  [3],  In  this  paper,  we  address  how  the  quality  of  the 
imagery  used  in  such  an  optical  system  affects  the  processors  performance.  The  data  bases 
we  used  include  multisensor  imagery  and  imagery  with  synthetic  reference  functions.  These 
data  are  part  of  the  DARPA  autonomous  terminal  homing  (ATH)  data  base.  In  Section  2,  we 
describe  the  optical  processor  and  address  the  issues  of  optical  processing  of  digitized 
data.  A  simple  low  pass  spatial  filtering  aperture  in  the  Fourier  transform  or  matched  spa¬ 
tial  filter  plane  is  shown  to  be  necessary  and  adequate  to  remove  the  effects  of  correlations 
due  to  the  fixed  regular  input  pixel  pattern  present  in  such  data.  In  Section  3,  the  dif¬ 
ference  between  system,  detector  and  correlation  noise  are  specifically  denoted  and  the  im¬ 
portance  of  the  system's  light  level  budget,  the  system's  diffraction  efficiency  n  and  the 
detector's  sensitivity  are  addressed.  Solutions  include  higher  power  light  sources  and  more 
sensitive  detectors.  However  preferable  solutions  are  increased  data  modulation  and  data 
space  bandwidth  product  and  the  use  of  bleached  or  dichromated  gelatin  matched  filters  (4). 


FIGURE  1.  Schematic  diagram  of  the  exper¬ 
imental  optical  scaling  frequency  plane 
correlator  used.  Code:  L  (laser ,  X»514nm); 
S (shutter):  VBS (variable  beam  splitter): 
M(mirror):  SFlpinhole  spatial  filter,  10X 
objective,  7um) :  CL (collimating  lens,  fL* 
762mm) ;  FTL (Fourier  transform  lens,  f 
762mn)  ;  P^dnput  plane);  ?2  (MSF  frequency 
plane);  P3 (correlation  plane). 


Optical  correlation  experiments  on  the  ATH  data  base  clearly  show  that  this  data  has  quite 
low  modulation,  spatial  frequency  content  and  space  bandwidth  product.  Fourier  transform 
plane  scans  and  our  initial  quantitative  analysis  in  Section  4  show  the  magnitude  of  this 
problem.  It  manifests  itself  as  a  quite  appreciable  output  correlation  peak  due  to  the  input 
aperture  with  a  smaller  peak  due  to  the  correlation  of  the  imagery  itself.  In  Section  5,  we 
advance  the  theory  and  experimental  verification  of  a  new  laboratory  technique  that  can  over¬ 
come  such  problems.  To  suppress  the  effects  of  the  aperture  correlation,  we  use  different 
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apertures  for  the  input  and  reference  functions.  This  technique,  together  with  the  high  pass 
filtering  action  of  the  MSF  are  shown  to  decrease  the  magnitude  of  the  aperture  correlations 
and  to  allow  detection  of  the  true  image  correlation  output. 

In  Section  6,  we  consider  optical  correlation  of  multisensor  ATH  data.  Both  digital  [5) 
and  optical  correlation  data  on  the  same  image  set  are  presented  and  compared  with  attention 
to  the  different  preprocessing  operations  possible  in  an  optical  and  digital  system  and  with 
attention  to  the  different  preprocessing  operations  required  for  multisensor  pattern  recog¬ 
nition.  We  find  that  in  most  practical  multisensor  pattern  recognition  cases,  the  optical 
and  digital  systems  perform  comparably  and  thus  conclude  that  in  the  majority  of  practical 
cases,  optical  processors  using  weighted  matched  spatial  filter  synthesis  are  adequate  for 
multisensor  pattern  recognition. 

In  Section  7 ,  we  turn  our  attention  to  the  ATH  data  base  containing  synthetic  reference 
objects.  These  are  found  to  differ  considerably  from  the  on-line  sensed  inputs  that  occur. 

In  this  case,  more  advanced  preprocessing  techniques  are  necessary  to  allow  high  quality  out¬ 
put  correlations  to  be  obtained.  Our  experiments  with  several  different  digitally  prepro¬ 
cessed  data  are  summarized.  We  find  the  gray  tone  modification  operator  to  perform  the  best, 
but  note  that  the  small  surface  lysing  preprocessing  operator  requires  more  analysis  and  that 
more  attention  to  modifications  of  both  operators  are  needed  when  the  number  of  independent 
objects  in  the  target  scene  to  be  recognized  increases. 

2.  OPTICAL  PROCESSING  OF  DIGITIZED  DATA 


Much  of  the  presently  available  image  data  bases  are  digitized.  The  ATH  data  base  repre¬ 
sents  one  of  the  most  complete  image  pattern  recognition  data  bases  available.  It  contains 
sensed  (real)  and  synthetic  imagery  with  various  differences  present  in  the  imagery  within 
a  given  set.  All  of  the  sensed  images  have  100  rows  and  ISO  columns,  whereas  the  synthetic 
images  are  150  x  150.  For  optical  processing,  these  data  are  recorded  on  film  (usually  in  a 
35mm  format) .  The  resultant  image  data  thus  has  fixed  horizontal  and  vertical  pixel  patterns 
(with  the  vertical  pattern  due  to  the  fixed  scan  line  separation  and  the  horizontal  pattern 
due  to  the  fixed  pixel  width) .  Care  must  be  taken  to  remove  the  fixed  image  pixel  pattern 
from  the  data,  else  it  will  produce  a  considerable  contribution  to  the  output  correlation. 

In  [61,  we  first  noted  this  problem  and  found  that  it  was  easily  overcome  by  placing  an  aper¬ 
ture  in  front  of  the  MSF  or  frequency  plane  to  block  the  spatial  frequency  components  corre¬ 
sponding  to  the  fixed  horizontal  and  pixel  spacings.  Since  the  spatial  frequency  correspond¬ 
ing  to  the  pixel  spacing  represents  the  highest  input  spatial  frequency  data,  they  can  easily 
be  filtered  out  without  degrading  the  image  information  (which  by  definition  must  lie  at  a 
lower  spatial  frequency) . 

The  optical  system  used  in  our  experiments  (Figure  1)  employed  weighted  MSF  synthesis  [2], 
This  concept  uses  the  fact  that  the  spatial  frequency  portion  of  the  data  to  be  emphasized 
can  be  controlled  by  picking  the  spatial  frequency  u 1  at  which  the  beam  balance  ratio  K  be¬ 
tween  the  intensities  of  the  plane  wave  reference  beam  and  the  signal  beam  equals  unity.  As 
K  is  adjusted  to  equal  unity  at  different  spatial  frequencies,  low,  medium  or  high  spatial 
frequency  data  in  the  input  and  reference  imagery  can  be  emphasized.  We  use  this  technique 
in  Section  5  to  perform  the  correlation  on  edge-enhanced  data  (to  decrease  the  effect  of  aper¬ 
ture  correlations).  In  Section  6,  the  same  technique  is  used  to  approximate  a  highpass  filter 
preprocessing  operator  for  multisensor  image  pattern  recognition.  In  Section  7,  the  same 
technique  is  again  used  for  similar  reasons  on  synthetic  reference  imagery. 

Our  ATH  experiments  reported  here  used  two  different  data  sets.  The  first  (Section  6) 
contained  imagery  of  the  Ames  building  obtained  with  3-4  different  sensors  at  three  different 
locations  either  downlooking  (0*)  or  target  looking  (from  a  depression  angle  of  20*  with 
respect  to  the  normal) .  The  second  data  set  (Section  7)  contained  synthetic  and  sensed 
images  of  a  Lockheed  building.  In  our  initial  autocorrelation  experiments,  K  was  set  equal 
to  one  at  spatial  frequencies  from  0.9-15cy/mm.  For  cross-correlations,  spatial  frequencies 
in  a  lower  0 . 86-6 . 6cy/mm  spatial  frequency  range  were  used.  In  our  multisensor  experiments, 
the  8-12um  sensed  images  were  used  as  references  since  they  correspond  to  a  good  spatial  fre¬ 
quency  band  in  which  significant  target  information  and  passive  detectors  exist.  This  band  is 
also  compatible  with  present  plans  for  the  ATH  system  that  include  use  of  an  8-12um  passive 
sensor  and  a  10.6um  active  CO2  laser  radar  (for  range  information).  The  multisensor  Ames 
image  data  had  quite  low  modulation  with  negligible  information  content  beyond  3cy/mm.  The 
synthetic  reference  images  (Sections  3  and  7)  were  of  even  lower  modulation  and  space  band¬ 
width  product,  necessitating  the  use  of  even  lower  spatial  frequency  settings  below  0 . 64cy/ram. 
To  allow  this  and  to  produce  optical  correlation  plane  data  whose  results  are  more  in  agree¬ 
ment  with  those  predicted;  by  theory,  we  photoreduced  the  sensed  and  synthetic  image  data 
bases  and  used  more  advanced  digital  preprocessing  algorithms  (Section  7) .  our  results  on 
these  data  were  in  agreement  with  theory. 


3.  NOISE  SOURCES  IN  OPTICAL  CORRELATORS 


3l0~25 


The  dynamic  range  of  a  digital  system  is  well-known  to  be  large.  But  moreso,  these  sys¬ 
tems  can  handle  low  modulation  data  by  proper  normalization  techniques.  Many  practical  prob¬ 
lems  that  arise  in  optical  systems  can  thus  be  ignored  in  digital  processing  such  as:  light 
source  intensity  levels,  filter  diffraction  efficiency,  and  detector  sensitivity.  As  our 
performance  measure  for  pattern  recognition,  we  use  the  output  correlation  SNR  defined  as 
the  ratio  of  the  intensity  of  the  correlation  peak  to  the  spatial  average  of  the  output  cor¬ 
relation  pattern  far  ( t > >0 )  from  the  correlation  peak  (assumed  to  occur  at  r»0) .  In  Figure 
2,  we  show  various  noise  sources  that  are  present  in  an  optical  correlator.  All  outputs  are 
measured  with  a  scanning  photometric  microscope  with  a  fiber  optic  probe  and  high  gain  low 
noise  PMT  interfaced  to  a  chart  recorder  on  which  the  correlation  plane  cross-sectional  scans 
were  obtained. 

For  the  system  under  test,  the  detector  noise  level  was  measured  with  no  input  light  inci¬ 
dent  on  the  detector.  It  thus  corresponds  to  the  detector  dark  current  and  defines  the  lowest 
output  intensity  that  the  experimental  system  can  detect.  The  source  of  this  noise  is  the 
PMT  used  to  detect  and  amplify  the  detected  light  signals.  The  scatter  level  of  the  system 
corresponds:  to  the  minimum  output  light  level  that  the  optical  system  can  support.  This 
noise  level  is  due  to  scatter  from  the  many  lenses,  mirrors  and  film  interfaces  present  in 
the  system.  This  measurement  was  obtained  with  the  MSF  in  place  (to  block  dc  and  low  spatial 
frequency  data)  and  with  a  film  of  uniform  transmittance  equal  to  the  average  transmittance 
of -the  input  image  in  place  at  the  input  plane  ?x.  This  scatter  level  should  be  above  the 
detector  noise  level.  This  requires  a  light  source  of  adequate  intensity  and  a  detector  of 
adequate  sensitivity. 
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FIGURE  2.  Laboratory  data  showing  three  types  FIGURE  3.  Cross-sectional  scan  of 

of  correlation  plane  noise.  the  Fourier  transform  of  a  synthetic 

reference  image  showing  its  low  modu¬ 
lation  and  low  frequency  content. 

The  correlation  pattern  contains  peak  sidelobes  and  other  structural  patterns  far  from 
the  peak.  These  latter  data  are  part  of  the  actual  true  correlation  and  we  refer  to  them  as 
correlation  noise.  To  insure  proper  measure  of  the  correlation  plane  SNR,  we  must  insure 
that  the  correlation  noise  level  is  above  the  lower  of  the  system's  scatter  level  and  the 
detector  noise.  If  the  source's  light  level,  the  input  data  modulation,  the  diffraction 
efficiency  of  the  MSF  or  the  detector's  sensitivity  is  too  low,  the  correlation  noise  will 
be  too  low  and  the  measured  correlation  plane  SNR  will  not  be  a  valid  measure  of  the  true 
SNR  of  the  output  data. 

For  the  data  in  Figure  2,  the  light  levels,  n ,  etc.  for  the  optical  system  measured  are 
adequate  (i.e.,  the  correlation  noise  level  is  above  the  scatter  level  and  the  detector 


noise).  Lower  detector  noise  levels  of  80dB  have  been  obtained  with  other  detectors  that  we 
have  (i.e.,  cooled  PMT  systems,  operated  in  a  photon  counting  mode)  Optical  systems  with 
better  components  have  been  assembled  by  us  and  have  produced  better  than  S5dB  scatter 
levels.  Thus,  the  data  in  Figure  2  should  be  viewed  only  qualitatively,  with  attention  to 
demonstrating  the  three  noise  sources  rather  than  with  attention  to  the  specific  noise 
values  listed. 


4.  DATA  MODULATION,  BIAS  AND  SPATIAL  FREQUENCY  EFFECTS 


From  our  remarks  advanced  in  Section  3  with  attention  to  the  three  noise  terms  in  Figure 
2,  the  importance  of  high  modulation  data  should  be  apparent.  In  Section  2,  we  noted  that 
the  synthetic  reference  imagery  was  of  low  modulation  and  spatial  frequency.  We  now  quanti- 
fy  these  issues.  To  do  this,  the  Fourier  transform  of  a  reference  image  was  optically  formed 
and  scanned  with  a  photometric  microscope.  The  Fourier  transform  of  the  Beet  function  aper¬ 
ture  was  also  obtained  optically  and  scanned  with  the  same  detector  system.  Subtracting 
these  two  patterns,  we  obtain  an  approximate  version  of  the  intensity  modulation  versus 
spatial  frequency  in  the  Fourier  transform  of  the  synthetic  reference  image  alone  (Figure  3). 
This  plot  shows  the  key  quantitative  features  of  the  synthetic  reference  imagery.  We  first 
note  that  the  image  has  no  useful  spatial  frequency  data  beyond  0.8cy/mm,  that  beyond 
0.3cy/ram  the  modulation  is  below  1%  and  at  no  spatial  frequency  is  the  modulation  above  10%. 
We  thus  see  from  this  representative  data  that  quite  severe  light  losses  will  result  when 
correlations  are  attempted  on  such  synthetic  reference  imagery.  We  return  to  this  issue  in 
Section  7 . 

Another  issue  associated  with  this  synthetic  reference  data  is  how  to  3et  It  •  1  at  spa¬ 
tial  frequencies  below  0.3cy/mm,  where  more  data  modulation  exists.  Since  the  diameter  of 
the  first  ring  of  the  wedge  ring  detector  used  to  set  the  K  beam  balance  ratio  [21  corres¬ 
ponds  to  a  spatial  frequency  of  0.38cy/mm,  setting  It  *  1  in  this  band  is  difficult.  This 
issue  was  further  complicated  because  scatter  from  the  dc  and  low  spatial  frequency  data  pre¬ 
cludes  accurate  settinq  of  K  *  1  at  such  low  spatial  frequencies  so  close  to  dc.  To  facili¬ 
tate  accurate  test  data  on  this  imagery,  we  photor educed  it  by  4.4X  to  increase  its  spatial 
frequency  content  and  we  rerecorded  the  data  on  high  contrast  film.  The  results  of  tests  on 
these  enhanced  imagery  are  presented  in  Section  7. 

To  provide  initial  quantification  of  the  effects  of  the  bias  level  b  and  the  modulation 
level  m  of  the  input  imagery  fix)  plus  the  effect  of  the  input  aperture  (assumed  for  now  to 
be  a  square  of  width  A) ,  we  describe  the  full  input  as 

t(x)  «  (b  +  mf  (x ))  Beet  (x/A)  .  (1) 

One-dimensional  functions  are  used  for  simplicity  with  no  loss  of  generality.  We  can  assume 
that  the  bandpass  filtering  action  of  the  MSF  removes  the  bias  level  and  thus  set  b  *  0  in 
(1).  A  similar  zero-mean  filtering  action  can  easily  be  provided  in  the  digital  processing 
of  the  data.  The  output  correlation  then  becomes 

c(x )  *  m2  f  (x)Rect(x/A)  0  f  (x)  Rect  (x/A) 

2  A/2-t 

»  ">  /  f  lx)f  (x  +  t)  dx  ,  (2) 

-A/2 

for  t  >.  0.  From  (2),  we  see  that  a  low  image  modulation  m  will  greatly  reduce  the  amplitude 
of  the  correlation  output  (as  is  well-known) .  We  also  note  that  the  input  aperture  A  will 
affect  the  variance  of  our  estimate  of  the  correlation  output  at  shifts  i>>0.  This  occurs 
because  less  overlapping  image  area  will  be  used  in  (2)  for  t>>0.  We  properly  normalized 
the  output  correlation  by  using 

2  A/2-t 

c(x)  *  s-t-tt  !  f<x)f(x  +  T»x  (3) 

A_|T|  -A/2 

for  t  >,  0.  This  corrects  the  optical  correlation  for  the  triangular  output  weighting  that 
would  normally  result  if  the  images  being  correlated  were  not  periodically  repeated  (as 
occurs  in  a  digital  circular  correlation).  However,  the  variance  of  the  correlation  noise 
estimate  will  still  be  worse  as  the  x  shift  is  increased  as  noted  in  conjunction  with  (2) . 

If  the  bias  level  b  were  not  completely  removed,  the  correlation  obtained  would  be 

C(T)  «  Ab2A(x/A)  (Am2/2)  (f  0  f)A  (x/A)  ,  (4) 

where  the  triangular  function  A(x)  »  1  -  x  for  |x|  <_  1  and  is  0  elsewhere.  From  (4),  we  see 
that  the  contribution  to  the  correlation  due  to  the  aperture  (first-term)  is  of  amplitude 
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A.b‘,  whereas  the  contribution  due  to  the  imaae  itself  is  Am2/2.  Assuming b  =  0.5  and  m  =  0 . 1 , 
the  contribution  to  the  correlation  due  to  the  aperture  is  Ib^/m^  =  2  ( 0 . 5) 2/ ( o  .  1 ) 2  =  50  times 
more  intense  than  the  imaae  correlation.  Suporession  of  the  dc  level  of  the  image  is  thus 
cuite  important  in  MSF  pattern  recognition. 

Our  optical  correlation  experiments  nave  shown  that  the  bias  level  b  is  not  completely 
removed  by  the  high  pass  filtering  action  of  the  MSF.  Thus,  the  actual  optical  correlation 
output  is  neither  (2)  nor  (4).  Rather,  it  is  similar  to  (4),  but  with  a  lower  bias  level  b 
than  the  one  present  in  the  original  imagery  of  (1) .  Moreso,  the  tlx)  input  is  high  pass 
filtered  or  differentiated  and  thus  the  factor  of  50  difference  in  the  aperture  and  image 
correlations  noted  above  will  be  greatly  reduced  in  practice.  However,  the  aperture  corre¬ 
lation  effect  cannot  be  neglected  if  m  is  low  as  we  show  in  Section  5.  An  exact  analysis  of 
this  issue  is  quite  image  dependent  and  appears  to  be  due  to  the  fact  that  a  local  digital 
edge-enhancement  operator  removes  the  mean  (or  bias  b)  of  each  part  of  the  image.  Conversely, 
the  optical  high  pass  filtering  operator  (being  a  global  rather  than  a  local  operator)  re¬ 
moves  the  average  bias  level  for  the  entire  image  and  thus  does  not  completely  remove  the 
bias  level  for  each  part  or  object  in  the  image. 

In  conclusion,  we  note  that  the  width  of  the  aperture  correlation  term  will  not  be  2A  as 
is  implied  by  (4) ,  but  rather  will  be  much  narrower  with  the  width  approximately  given  by 
twice  the  reciprocal  of  the  bandwidth  of  the  data.  This  is  important,  since  the  image  corre¬ 
lation  is  thus  not  masked  within  the  wide  aperture  correlation  function.  Rather,  it  is  quite 
well  separated  (see  Figure  4a  and  Section  5) . 

5.  APERTURE  CORRELATION  EFFECTS 


From  Section  4,  we  have  shown  the  contribution  due  to  the  optical  output  correlation  can 
be  significant  when  the  image  modulation  is  low  (as  it  is  in  Figure  3) .  We  also  note  that 
the  global  versus  the  local  high  pass  filtering  operation  of  the  weighted  optical  MSF  does 
not  remove  the  bias  level  of  each  object  in  the  image  and  thus  leaves  a  potentially  large 
contribution  due  to  the  aperture  correlation.  We  now  consider  techniques  to  reduce  these 
effects . 


FIGURE  4.  Output  correlation  plane  pattern  showing  (a)  both  the  aperture 
and  image  correlations  and  (b)  only  the  image  correlation  (obtained  after 
high  pass  filtering  and  use  of  different  apertures  for  the  mout  and  ref¬ 
erence  imagery) . 


We  describe  the  reference  image  to  be  used  as 

r(x)  *  (b  »  mf  (x)]  Circ  (x)  (5a) 

and  the  input  image  as 

t(x)  *  [b  +  mf(x  *  Xq ) J  Rect (x) ,  (5b' 

where  the  input  is  assumed  to  be  shifted  by  Xq  from  the  reference  location  and  where  differ¬ 
ent  aperture  functions  are  used  for  the  input  and  reference  images.  It  is  possible  to  place 
both  the  input  and  reference  functions  in  rectangv'jr  apertures  of  different  widths  Aj  and 


J 


z; 


A2  or  to  use  different  aperture  functions  directly.  In  our  present  description,  we  assume 
circular  and  rectangular  apertures  for  the  reference  and  input  imagery,  since  this  was  the 
actual  procedure  used  in  the  laboratory.  As  noted  earlier,  high  pass  filtered  versions  r '  (x) 
and  t1  (x)  will  be  correlated  in  the  actual  optical  MSF  system.  The  correlation  r'Qt1  will 
thus  contain  four  tents.  Two  are  the  average  of  f  over  the  aperture  regions.  With  zero- 
mean  imagery,  these  terms  are  negligible.  The  remaining  terms  are  the  aperture  correlation 
(located  at  a  fixed  position  r  =  0)  and  the  correlation  of  the  high-pass -f iltered  image  ? 
(Located  at  t  =  t0  «  xg)  . 

To  describe  the  high-pass-filtered  data,  we  aoproximate  [7]  the  optical  high  pass  filter  (HPF) 
function  as  a  derivative  filter  with  a  bandwidth  BW  equal  to  that  of  the  signal  f  (x) .  The 
edges  in  the  differentiated  data  will  thus  have  a  small  but  finite  width  (1/BW) .  For  the 
case  of  the  Sect  aperture,  we  write  the  derivative  function  as 

[ Rect (x/A) ] '  *  Rect[ (x  -  A/21BW]  +  Rect[(x  +  A/2) BW] ,  (6) 

or  as  two  pulses  of  width  1/BW  located  at  +  A/2.  The  correlation  of  r'  and  t'  now  contains 
25  terms.  However  many  can  be  eliminated  "and  the  resultant  expression  simplified  as  we 
now  discuss.  We  write  the  terms  containing  f(x)  times  the  derivative  of  the  aperture  func¬ 
tion  as 


f (x) d (Aperture) /dx  =  kd (Aperture) /dx .  (7) 

This  is  realistic,  since  the  derivative  of  the  aperture  function  is  a  circular  or  rectangular 
edge  (in  2-D)  of  width  1/BW  as  in  (6)  and  the  image  function  f (x)  only  effects  the  amplitude 
of  this  edge  at  the  locations  where  the  two  overlap.  We  represent  this  factor  by  the  cons¬ 
tant  k  term  in  (7) .  The  correlation  output  terms  can  thus  be  grouped  as 

C(t)  -  [b2  +  2bkm  +  k2m2]  (Rect*  (x)  ©Circ'  (x)  ] 

+  (bm  +  km2] ( Sect (x) f 1 (x)  Q  Circ 1 (x) ] 

+  [bm  +  km2  ]  [Rect  ’  (x)  ©  Circ  (x)  f '  (x)  ] 

+  m2 [ Rect  (x)  f  '  (x)  0  Circ  (x)  f  '  (x)  ]  (8) 

The  second  and  third  terms  in  (8)  are  negligible  because  f’(x)  is  of  zero-mean.  The  first 
term  will  also  be  negligible  since  the  correlation  of  the  derivatives  of  two  different  aper¬ 
tures  will  have  very  little  in  common  (i.e.,  for  the  case  of  the  circular  and  square  aper¬ 
ture  of  equal  diameter  and  width,  their  derivatives  only  overlap  at  the  four  areas  in  the 
center  of  each  edge  of  the  square  aperture) .  The  resultant  correlation  thus  reduces  to  the 
fourth  term  in  (8).  Assuming  approximately  equal  areas  for  the  two  apertures,  we  thus  write 
the  correlation  output  pattern  as  (with  m  =  1) 

C(r)  =  (  f  (x)  ©  f  •  (x)  ]  .  (9) 

In  Figure  4a,  we  show  the  correlation  output  pattern  in  (4)  with  the  same  Rect  aperture 
function  used  for  the  input  and  reference  imagery.  The  natural  HPF  action  of  the  optical 
MSF  is  included.  This  reduces  the  aperture  correlation  term  in  (4) ,  but,  as  seen,  the  aper¬ 
ture  correlation  peak  on  the  left  is  still  quite  appreciable  compared  to  the  image  correla¬ 
tion  peak  on  the  right.  (The  input  image  was  displaced  by  xg  to  show  these  two  correlation 
peaks  separately).  In  Figure  4b,  we  show  the  correlation  output  pattern  in  (8)  using  dif¬ 
ferent  apertures  for  the  input  and  reference  imagery.  The  output  is  the  single  correct  image 
correlation  peak  as  predicted  by  (9) . 

6.  COMPARISON  OF  OPTICAL  AND  DIGITAL  MULTISENSOR  IMAGE  CORRELATIONS 


As  our  first  experimental  optical  correlations,  we  considered  the  processing  of  various 
pairs  of  multisensor  images  of  the  same  scene  (Ames)  taken  in  different  wavelengths  (see 
columns  1  and  2  of  Table  1).  In  multisensor  image  pattern  recognition,  the  edges  of  the 
objects  in  the  scene  are  well-known  to  be  most  useful,  reliable  and  reproducable  image  parts 
on  which  to  base  a  pattern  recognition  correlation.  Thus,  various  edge-enhancement  prepro¬ 
cessing  techniques  have  been  used  in  multisensor  image  detection  and  registration.  In  [5], 
we  have  explored  multisensor  image  detection  in  depth.  Our  present  concern  is  to  assess  the 
use  of  different  edge-enhancement  preprocessing  operators  for  multisensor  pattern  recognition 
and  to  compare  the  results  of  optical  and  digital  correlations  on  multisensor  image  pairs. 


Table  1.  Optical  and  digital  multisensor  correlations  (Ames  data  base) 
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In  column  four  of  Table  1,  we  show  the  results  of  a  direct  digital  correlation  on  selected 
multisensor  image  pairs  (with  no  preprocessing) .  These  should  be  compared  to  the  results  in 
column  five  using  optical  weighted  MSF  correlations  (with  the  K  «  1  spatial  frequency  setting 
with  the  best  correlation  output  used) .  As  our  performance  measure,  we  use  SNR  (defined  as 
the  ratio  of  the  correlation  peak  to  the  variance  of  the  correlation  far  from  the  peak) .  We 
note  that  the  optical  correlation  results  are  comparable  to  those  of  the  digital  correlator 
with  better  results  obtained  optically  for  most  correlation  pairs  and  slightly  poorer  results 
obtained  for  only  two  multisensor  correlations.  This  is  due  to  the  fact  that  the  optical 
correlator  always  produces  a  high  pass  or  band  pass  filtering  action,  whereas  the  full  image 
spectrum  is  used  in  the  digital  correlations.  The  two  cases  when  better  SNR  resulted  from 
digitally  processed  data  correspond  to  the  case  of  an  autocorrelation  and  to  an  image  pair 
for  which  the  correlation  measure 


CM 


/Ir(x)Is(x)  dx 
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(10) 


was  large  (0.81).  This  correlation  measure  in  (10)  is  the  correlation  at  registration  nor¬ 
malized  by  the  energy  in  the  two  images.  As  shown  in  [5),  large  values  of  CM  indicate  that 
the  image  pairs  have  significant  common  information  and  thus  that  they  are  unisensor  moreso 
than  multisensor  data.  In  such  cases,  HPF  preprocessing  will  degrade  the  output  correlation 
as  noted  in  [5].  Thus,  both  cases  in  Table  1  for  which  optical  HPF  processing  proved  infer¬ 
ior  correspond  to  unisensor  not  multisensor  data. 

To  provide  further  insight  into  multisensor  pattern  recognition  and  our  comparison  of 
optical  and  digital  correlations,  a  |GRAC|  preprocessing  operator 


|  GRAD | 


(11) 


was  applied  to  both  images.  The  subscripts  i,j  in  (11)  refer  to  image  pixel  numbers.  The 
correlation  results  are  shown  in  column  6.  A  digital  HPF  preprocessing  operation  with  vari¬ 
ous  cutoff  frequencies  was  also  applied  to  two  of  the  multisensor  image  pairs.  The  results 
for  the  best  cutoff  frequency  are  given  in  column  7.  Comparing  the  digital  | GRAD j  operator 
(column  6)  with  the  optical  HPF  (column  5),  we  see  that  the  j  GRAD  I  performs  better  in  all 
cases  except  the  last  one.  From  [5],  we  know  that  the  |GRAD|  operator  whitens  the  image 
spectrum  and  equalizes  the  polarity  of  the  edges  in  the  edge-enhanced  image.  The  improvement 
for  the  autocorrelation  of  22  @  22  is  due  to  whitening  of  this  uni-sensor  case.  The  loss 
for  the  10(*)ll  case  is  due  to  the  high  CM  for  this  pair  of  images.  The  two  central  correla¬ 
tions  noted  in  Table  1  are  typical  of  most  of  the  results  we  obtained.  They  verify  that  for 
multisensor  imagery  (low  CM) ,  the  absolute  value  operator  improves  the  output  correlation 
SNR  over  that  obtained  with  an  optical  or  digital  HPF  (column  5  and  column  7) .  Comparing 
columns  5  and  7,  we  3ee  that  the  digital  HPF  gives  a  better  SNR.  This  is  due  to  the  fact 
that  a  much  lower  cutoff  frequency  (0,23cy/mm)  was  possible  with  a  digital  HPF  compared  to 
the  optical  HPF  (0.86cy/mm)  due  to  dc  scatter  and  low  image  modulation. 

Despite  these  differences,  it  appears  that  an  optical  weighted  MSF  is  adequate  for  most 
multisensor  pattern  recognition  (with  low  CM  <  0.5  values)  if  K  can  be  set  equal  to  unity  at 
a  low  enough  spatial  frequency  band  (see  Section  7).  In  Figure  5,  we  show  two  of  the  multi¬ 
sensor  images  (22  and  2S)  and  the  resultant  real-time  optical  output  correlation. 


(5b)  \  »  0.8-0.95um 


(Sc)  Correlation  (6c)  Correlation 


FIGURE  5.  Input  multisensor  imagery  (a)  FIGURE  6.  Correlation  (c)  of  sensed  (a! 
and  !b)  and  optical  output  correlation  and  synthetic  (b)  image  data  after  gray 
plane  pattern  (c) .  tone  modification  ( GTM )  preprocessing. 
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7.  CORRELATION  WITH  SYNTHETIC  REFERENCE  FUNCTIONS 


As  noted  in  Section  4  and  as  shown  in  Figure  3,  the  modulation  and  spatial  frequency  con¬ 
tent  of  the  synthetic  reference  imagery  was  quite  Iovj.  To  allow  optical  correlations  on  these 
data  with  measureable  SNRg  values  (with  the  correlation  noise  level  above  the  system’s  scat¬ 
ter  level,  see  Figure  2)  and  with  the  beam  balance  ratio  set  equal  to  one  at  a  sufficiently 
low  spatial  frequency  (see  Figure  3  and  Table  1) ,  it  was  necessary  to  photoreduce  the  data 
and  to  apply  more  sophisticated  preprocessing  operators.  In  Figure  6,  we  show  typical  ref¬ 
erence  and  sensed  incut  images  (Lockheed  building) .  Differences  in  the  contrast,  detail  on 
the  roof  and  in  image  noise  (e.g.  trees  and  cars  are  present  in  the  sensed  image,  but  are 
absent  in  the  synthetic  reference  image)  are  obvious. 

We  first  photoreduced  the  sensed  and  reference  images  in  this  data  set  by  a  factor  of  4.4. 
This  effectively  increased  all  spatial  frequencies  in  the  data  by  a  factor  of  4.4.  (Recall 
from  Section  6  and  Table  1  that  the  optimal  spatial  frequency  cutoff  found  from  the  digital 
correlations  on  the  multisensor  data  was  0.23cy/mm,  whereas  our  optical  system  could  not  be 
used  below  0.86cy/mm.  Also  note  from  Figure  3  that  the  synthetic  reference  imagery  has  neg¬ 
ligible  modulation  beyond  0.8cy/mm).  The  photoreduction  performed  thus  allows  K  to  be  set 
equal  to  one  in  the  more  appropriate  lower  spatial  frequency  bands.  With  this  step,  we  ex¬ 
pect  experimental  results  that  agree  with  theory. 

As  new  preprocessing  operators,  we  use  gray-tone  modification  (GTM)  and  small-surface 
lysing  (SSL).  These  digital  preprocessing  operations  [81  were  performed  by  Technology  Ser¬ 
vices  Corporation,  who  provided  us  with  35mm  transparencies  of  the  sensed  and  synthetic  tar¬ 
get  imagery  with  and  without  these  preprocessing  operators  applied.  The  GTM  operation  cal¬ 
culates  the  number  of  pixels  in  the  sensed  and  reference  images  with  given  gray  level  values. 
It  uses  this  histogram  to  equate  gray  levels  in  the  two  images  with  approximately  equal  dis¬ 
tributions.  It  also  performs  histogram  equalization  [91  to  allot  the  optimal  portion  of  the 
gray  level  range  available  to  those  image  regions  with  appropriately  large  distributions  for 
a  given  gray  level.  Latching  is  used  to  provide  a  good  mapping  or  histogram  of  the  gray 
levels  in  the  imagery.  This  preprocessing  operator  thus  makes  the  gray  levels  of  correspond¬ 
ing  regions  of  the  sensed  (input)  and  reference  (synthetic)  imagery  equal  plus  increases  the 
contrast  and  modulation  of  the  imagery.  From  our  discussions  in  Sections  3,  4  and  5,  these 
are  both  desirable  image  attributes  for  optical  correlation.  The  SSL  preprocessing  operator 
is  used  to  remove  small  surfaces  that  are  not  connected  to  larger  ones.  This  is  most  useful 
in  removing  the  noise  produced  by  edge  enhancement.  Thi3  operation  is  achieved  by  a  smart 
median  replacement  of  all  image  pixel  levels.  This  is  achieved  by  first  determining  thresh¬ 
olds  for  the  normalized  magnitude  of  an  edge  and  the  correlation  length  of  the  image  tex¬ 
ture  from  a  training  set. 

Our  experimental  data  obtained  on  the  down-looking  ATH  data  base  for  the  Lockheed  imagery 
are  summarized  in  Table  2.  In  columns  1  and  2  ,  we  provide  the  results  of  correlations  with 
a  synthetic  reference  function  with  a  high  u*  spatial  frequency  band  setting.  In  columns  3 
and  4,  we  provide  analogous  data  for  the  GTM  preprocessed  synthetic  reference  with  a  much 
lower  u’  spatial  frequency  cutoff  band  emphasized.  The  different  entries  in  each  column  cor¬ 
respond  to  different  preprocessed  inputs  (SYN  ■  synthetic,  SEN  »  sensed,  GTM/SSL  refers  to 
data  with  both  the  GTM  and  the  SSL  operators  applied) .  The  output  correlation  plane  SNR 
values  for  each  case  are  denoted  in  columns  2  and  4  for  comparison. 

In  these  optical  correlation  experiments,  the  use  of  photoreduced  imagery  allowed  lower 
u'  settings  to  be  used  and  hence  more  reliable  results  to  be  obtained.  This  was  possible 
because  the  effects  of  scatter  from  dc  spatial  frequency  data  was  decreased  with  the  increased 
image  spatial  frequency  provided  by  the  photoreduction.  All  SNR  values  with  the  GTM  pre- 
processed  synthetic  reference  (column  4)  exceed  those  obtained  with  the  original  unprepro- 
cessed  synthetic  reference  (column  2).  As  row  1  in  Table  2  shows,  the  GTM  operator  improves 

the  SNR  of  the  autocorrelation  (due  to  the  increased  data  modulation  provided) .  As  row  2 

shows,  better  cross-correlations  were  obtained  with  the  GTM  preprocessed  reference  and  even 
better  results  (column  4)  with  a  GTM  preprocessed  sensed  image  also.  The  SSL  operator  alone 
(row  5)  gave  poor  results  and  when  used  in  conjunction  with  the  GTM  operator  (row  4),  the 

results  obtained  were  inferior  to  those  of  the  GTM  operator  alone.  These  data  lend  credence 

to  further  studies  of  these  and  other  more  advanced  preprocessing  operators.  The  decrease 
in  performance  obtained  when  the  SSL  operator  was  used  (even  if  used  in  conjunction  with  the 
GTM  operator)  indicate  that  more  analysis  and  attention  should  be  given  to  the  details  of 
this  operation  (i.e.  any  image  distortion  it  introduces,  any  image  modulation  decrease  it 
produces,  and  any  true  image  data  it  removes).  Additional  es.-eriments  performed  with  the 
SSL  operator  on  data  with  a  large  number  of  independent  objects  indicates  that  the  latter 
issue  may  be  of  major  concern  in  future  work.  In  Figure  6c,  we  show  the  optical  correlation 
output  obtained  from  the  given  sensed  and  synthetic  reference  functions  shown  in  Figures  6a 
and  6b . 


Table  2.  Correlations  with  synthetic  reference 
imagery  using  various  preprocessing  operators 
(Lockheed  database) . 
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8. _ SUMMARY  AND  CONCLUSION 


In  this  paper,  various  aspects  of  image  quality  and  its  effects  on  the  output  correlation 
from  an  optical  weighted  matched  spatial  filter  correlator  were  addressed.  The  fixed  pixel 
pattern  of  the  input  using  digitized  data  was  noted  and  the  effect  removed  by  the  use  of  a 
simple  spatial  frequency  in  the  transform  plane.  Three  sources  of  optical  correlation  plane 
noise  were  then  noted  and  the  effective  data  modulation,  spatial  frequency  and  input  aper¬ 
ture  effects  were  discussed  for  each.  Low  spatial  frequency  data  often  precludes  setting  K 
at  sufficiently  low  spatial  frequencies.  Use  of  photoraduced  imagery  was  shown  to  overcome 
this  problem.  Synthetic  reference  imagery  was  found  to  have  low  modulation  and  very  low 
spatial  frequency  data.  The  correlation  of  this  reference  data  was  found  to  be  possible  with 
the  use  of  advanced  digital  preprocessing  operators  such  as  gray-tone  modification  and  small 
surface  lysing.  Aperture  correlations  were  found  to  dominate  all  image  correlations.  This 
effect  was  overcome  by  the  use  of  different  apertures  for  the  input  and  reference  function 
together  with  high  pass  filtering.  A  comparison  of  optical  and  digital  correlations  on 
multisensor  imagery  showed  that  the  high  pass  filtering  action  performed  by  the  weighted 
matched  spatial  filter  correlator  was  appropriate  for  the  majority  of  multisensor  image  pat¬ 
tern  recognition  problems  (this  class  of  multisensor  imagery  was  found  to  be  measureable  by 
a  correlation  measure,  the  energy  normalized  correlation  at  registration) .  From  these 
studies,  much  new  insight  on  the  effect  of  image  quality  on  the  performance  of  an  optical 
matched  spatial  filter  correlator  have  been  determined  together  with  the  use  of  weighted 
matched  spatial  filter  high  pass  filtering  for  multisensor  pattern  recognition  and  the  need 
for  advanced  preprocessing  techniques  in  correlations  with  synthetic  reference  imagery. 
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ABSTRACT 

A  general  multisensor  image  processor  and  image  model  are  described  and  experimental  veri¬ 
fication  of  their  use  are  provided.  The  pixel  correlation  coefficient  is  shown  to  be  an 
adequate  measure  of  the  statistical  independence  of  the  gray  levels  in  a  multisensor  image 
pair  and  to  be  of  use  as  a  guide  to  selecting  various  preprocessing  operators.  Experimental 
results  obtained  on  the  various  multisensor  image  data  bases  verify  our  theory  quite  well 
and  have  contributed  considerable  practical  insight  into  the  processing  of  imagery  with 
random  contrast  reversal  intensity  differences. 

1 .  INTRODUCTION 


In  nearly  all  practical  pattern  recognition  and  image  registration  applications,  the  in¬ 
put  and  reference  images  can  differ  in  a  variety  of  ways  and  for  many  diverse  reasons.  In 
this  paper,  we  consider  the  registration  or  recognition  of  images  of  the  same  scene  when  the 
images  originate  from  different  sensors  or  from  the  same  sensor  under  different  conditions. 
The  latter  case  arises  in  infrared  imagery  and  can  also  be  due  to  differences  in  weather, 
seasons,  sun  angle,  time  of  day,  etc.  We  term  all  such  data  as  multisensor  imagery  (MSI) . 

We  distinguish  MSI  from  unisensor  imagery  (USI)  data  by  noting  that  in  MSI  data,  the  gray 
level  of  the  same  object  in  different  images  is  independent.  When  total  independence  exists, 
we  refer  to  the  data  as  pure  MSI.  When  total  dependence  occurs,  we  classify  the  data  as  USI. 

Much  MS  imagery  lies  between  the  extremes  of  pure  MSI  and  pure  USI  data.  As  our  measure 
of  the  class  of  imagery,  we  use  the  pixel  correlation  coefficient  for  a  reference  Ir  and 
sensed  I,  image  pair, 

s  fl  (x)I  (x)  dx 

o  «  - - - - - 177  ’  (1> 

(/l/(x)  dx/Igfx)  dx] * 

cne-dimensicnal  functions  are  used  for  simplicity  and  the  correlation  peak  is  asssumed  to  occur 
at  the  origin  with  no  loss  in  generality.  The  ratio  in  (1)  of  the  intensity  of  the  corre¬ 
lation  at  registration  to  the  product  of  the  energies  in  the  two  images  is  thus  a  useful 
normalized  measure  of  how  well  they  correlate.  When  p  «  1,  the  imagery  is  pure  USI.  Where¬ 
as,  when  a  »  0,  the  data  is  pure  MSI. 

As  our  performance  measure,  we  select  the  output  signal-to-noise  ratio  SNRq  in  the  corre¬ 
lation  plane,  defined  by 
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The  numerator  in  (2)  is  the  expected  value  of  the  correlation  at  registration  t  =  0  (with 

the  expectation  taken  over  all  scenes  and  conditions).  The  denominator  is  the  standard  devi¬ 
ation  of  the  output  correlation  far  from  the  correct  registration  point.  This  SNR  is  easily 
measured  in  the  laboratory  and  it  can  be  related  to  other  SNR  measures  as  well  as  to  the 
probability  of  detection  Pp,  the  probability  of  false  alarm  Pp^,  the  probability  of  correct 
registration  pc  and  the  probability  of  error  Pp.  It  is  thus  a  most  appropriate  parameter 
to  use  in  comparing  various  image  registration  systems. 


A  wealth  of  work  in  MSI  registration  exists.  However,  in  some  cases  (e.g.  [14]),  the 
data  used  does  not  exhibit  random  contrast  reversals.  Moreso,  no  noise  was  present  in  sev¬ 
eral  of  the  cases  analyzed  (e.g.  (151).  Thus,  such  cases  actually  typify  USI  data  rather 
than  MSI  data.  Moreso,  no  particular  preprocessing  operations  or  techniques  are  necessary 
to  register  such  imagery.  In  our  work,  we  are  concerned  with  pure  MSI  data,  with  those 
image  features  that  give  rise  to  such  data,  and  with  the  optimal  processor  to  be  used  for 
such  data  recognition  problems.  Many  heuristic  methods  have  been  advanced  and  used  to  reg¬ 
ister  MS  data.  These  involve  optical  (1,2)  and  digital [3-5]  systems,  and  edge  detection 
techniques  [6,7[  and  template-matching  methods  [8,9] .  The  most  successful  results  have  been 
obtained  using  gradient  edge  detection  preprocessing  operators  followed  by  correlators  [3,10- 
121. 


Our  work  makes  use  of  this  prior  art.  However,  we  envision  a  hybrid  optical-digital 
processor  in  which  the  preprocessing  operations  are  performed  digitally  and  the  correlation 
is  performed  optically.  We  also  adopt  a  more  fundamental  and  basic  approach  to  this  MS  reg¬ 
istration  problem  that  includes  a  supporting  image  model  and  a  general  MSI  processor  archi¬ 
tecture.  This  model  and  general  processor  "[13]  are  summarized  in  Section  2.  The  highlights 
of  our  statistical  analysis  of  this  processor  were  provided  earlier  [13].  In  this  paper,  we 
thus  concentrate  on  the  experimental  verification  of  these  results.  In  Section  3,  we  first 
show  that  conventional  correlation  cannot  work  for  MSI  data  and  we  advance  a  communication 
analogy  for  this  problem.  Our  results  provide  considerable  insight  into  the  nature  of  the 
MSI  registration  problem.  Following  a  description  of  the  MSI  data  bases  used  (Section  4) , 
experimental  data  on  six  aspects  of  MSI  registration  are  advanced.  These  include:  the  use 
of  o  as  a  measure  of  the  type  of  imagery  (Section  5) ,  how  the  number  of  objects  in  the  image 
affects  the  result  (Section  6),  the  need  for  Gaussian-histogram  reshaping  (Section  7),  veri¬ 
fication  of  our  MSI  model  (Section  S) ,  a  comparison  of  high  pass  filtering,  gradient  and 

image  whitening  operators  (Section  9)  and  threshold  selection  when  noise  is  present  in  the 

data  (Section  10) . 

2.  IMAGE  MODEL  AND  MULTISENSOR  PROCESSING  ARCHITECTURE  [13] 

We  define  an  object  as  a  region  with  a  homogeneous  gray  level,  separated  from  other  ob¬ 
jects  by  edges.  Although  the  gray  levels  of  the  sane  object  in  different  MSI  are  statisti¬ 

cally  independent  random  variables,  the  edges  of  the  objects  are  preserved  in  the  MS  imaging 
process.  In  our  mathematical  model  of  pure  MSI  data,  the  scenes  to  be  registered  are  assumed 
to  contain  objects  larger  in  extent  than  several  pixels.  We  assume  zero-mean  data.  This  is 
directly  realized  in  optical  matched  spatial  filter  processors.  In  digital  processing,  this 
assumption  is  equivalent  to  removing  the  mean  of  the  data.  Since  the  mean  of  the  data  does 
not  provide  reliable  information,  such  an  assumption  is  practical  and  necessary. 

A  cross-sectional  scan  through  a  MS  image  is  thus  assumed  to  yield  a  train  of  pulses 
whose  amplitudes  and  locations  are  random  variables.  The  location  of  the  pulses  (image 
edges)  are  assumed  to  have  a  distribution  that  corresponds  to  a  low-density  Poisson  statis¬ 
tical  distribution  with  an  average  pulse  rape  X  per  unit  distance .  The  amplitudes  of  the  pulses 
are  assumed  to  have  zero-mean  Gaussian  density  functions.  We  assume  white  zero-mean  Gaussian 
noise  to  be  added  to  the  sensed  image.  We  also  assume  the  pulse  trains,  their  amplitudes  and 
noise  to  be  stationary  and  statistically  independent. 
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FIGURE  1  General  MSI  Pattern  Recognition  Processor 


Our  general  (IS  processor  model  considers  five  preprocessing  operators  (applied  to  both 
the  reference  Ir  and  the  sensed  Is  images)  followed  by  a  correlator  as  shown  in  Figure  1. 
Spectrum  whitening  refers  to  producing  data  spectra  that  are  white  (i.e.  with  the  Fourier 
transforms  of  the  image  plus  noise  being  uniform  for  both  the  sensed  and  reference  imagery) . 
Since  the  edges  of  the  objects  are  the  common  features  that  are  preserved  in  MS  image  pairs, 
edge-enhancement  is  a  necessary  preprocessing  function  for  MSI  registration.  However,  since 
the  gray  levels  of  all  objects  in  an  image  are  random,  then  so  are  the  polarities  of  all  of 
the  edges  in  an  edge-enhanced  image.  We  thus  also  consider  a  simple  nonlinear  absolute  oper¬ 
ator  and  its  use  in  equalizing  the  polarity  of  the  edges  in  an  edge-enhanced  image  pair,  we 
will  consider  various  edge-enhancement  operators  separately.  However,  experiments  have  shown 
(Section  3)  that  edge-enhancement  operators  generally  also  provide  approximate  whitening 
operations.  Thus,  in  practice,  both  of  these  preprocessing  operators  are  included  as 

one  rather  than  separately.  We  find  the  gradient  operator  to  work  quite  well  and  thus  we 
compare  the  need  for  polarity  equalization  by  applying  the  signed  gradient  operator  (SGN) 
and  the  absolute  value  |GRAD|  gradient  operator  to  the  same  image  pair  and  comparing  the 
resultant  output  correlation  SNRq.  We  use  the  approximate  iGRADi  operator 
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for  the  absolute  value  of  the  gradient  at  pixel  (i,j)  of  image  I  because  it  is  easily  real¬ 
ized  digitally  and  because  experimental  verification  indicates  that  this  operator  is  quite 


useful . 


A  histogram  reshaper  is  the  next  preprocessing  operator  we  consider.  To  use  a  correlator 
as  the  final  image  registration  step,  the  histogram  of  the  pixel's  gray  levels  in  the  imagery 
should  theoretically  be  Gaussian.  Forming  the  magnitude  of  the  gradient  of  an  image  is  a 
quite  nonlinear  operation  that  is  expected  to  signif icantly  change  the  pixel's  histogram. 

We  consider  the  importance  of  the  pixel's  histogram  distribution  by  applying  the  nonlinear 
mapping 


Y  «=  log  (X  +  1) 


(4) 


to  the  histogram  of  the  ordered  pixel  gray  levels  X.  This  histogram  reshaping  function  was 
found  empirically  to  approximately  produce  the  Gaussian  histogram  theoretically  needed  for 
a  correlation  image  registration  processor.  The  threshold  operator  is  applied  to  the  pixel 
intensity  levels  X  to  reduce  additive  noise  in  the  data  prior  to  correlation.  This  simply 
involves  replacing  pixels  with  intensity  levels  X  Th  and  X  >  Th  by  new  values  Y  such  that 


Y 


0  ,  X  <.  Th 

X-Th  ,  X  >  Th  ' 


(5) 


In  Sections  5  -  10 ,  we  present  the  results  of  various  experimental  studies  of  these  dif¬ 
ferent  preprocessing  operators  and  other  issues  associated  with  the  image  model  and  general 
multisensor  processor.  For  example,  we  consider:  the  Poisson  pulse  rate  X,  number  of  ob¬ 
jects  present  in  the  image,  threshold  selection,  input  SNRi,  other  edge-enhancement  opera¬ 
tors,  and  image-whitening  operators.  Theoretical  analyses  of  the  image  model  and  the  MS 
processor  architecture  were  advanced  earlier  by  us  [13].  Thus,  our  present  emphasis  is  on 
the  experimental  verification  of  the  predicted  results. 


3.  USI  AND  MSI  PROCESSING 

To  provide  insight  into  the  differences  between  USI  and  MSI  data  and  the  processing  nec¬ 
essary  to  register  each,  we  highlight  the  results  of  a  statistical  analyses  of  conventional 
correlation  (with  no  preprocessing)  applied  to  pure  MSI  data.  We  also  advance  a  communica¬ 
tions  analogy  by  which  the  potential  need  for  the  various  preprocessing  operators  included  in 
our  MS  processor  model  of  Figure  1  can  be  seen.  For  computational  reasons,  we  use  the  cor¬ 
relation  SHR,  defined  as  the  ratio  of  c(0)  to  the  standard  deviation  of  c(t)  for  t  »  0  as 
ouv  performance  measure.  Applying  standard  statistical  assumptions  about  the  data,  it  is 
quite  easy  to  show  that  both  c(0)  and  c(»)  are  zero-mean  Gaussian  random  variables  with 
standard  deviations  of  ag//n  and  ag/*Tcn  respectively.  In  this  notation,  aQ  is  the  standard 
deviation  of  an  object  of  area  equal  to  that  of  the  full  image,  n  is  the  number  of  indepen¬ 
dent  objects  in  a  given  image  and  kn  (where  k  >  1)  is  the  number  of  apparent  objects  ores- 
ent  in  the  overlap  of  two  MS  images  far  (x  ■  *)  from  registration.  In  practice,  og/f'En  is 
a  lower  bound  on  the  correct  standard  deviation. 


Replacing  the  standard  deviation  by  its  average,  or  the  statistical  variance,  the  SHRg  is 
seen  to  be  a  zero-mean  Gaussian  random  variable  with  standard  deviation  /1c.  Thus,  for  any 
desired  SNRg  »  Th,  Pq  is  non-zero  and  Pp  increases  as  k  increases  (  and  hence  as  the  number 
n  of  independent  objects  in  the  image  increases).  For  k  ■  4,  we  find  Pd  <  10~6 for  even  a  low 
desired  SNRg  of  10.  The  problem  of  correct  registration  Pc  was  also  derived  for  this  case. 
We  found  that  for  n  «  50  and  k  «  3.2,  Pc  was  *  10" 2 .  Moreso,  we  showed  that  a  random 

choice  of  any  correlation  peak,  rather  than  selection  of  the  largest  one,  gave  a  better  Pc 

value.  Thus,  conventional  correlation  on  MSI  data  is  seen  to  yield  quite  poor  performance. 

Whitening  the  spectrum  of  imagery  improves  the  performance  of  USI  correlations,  but  is  of 
no  use  in  MSI  correlation  because  it  simply  increases  the  amplitude  of  all  peaks  in  the  cor¬ 
relation  plane  (the  correct  peak  as  well  as  all  sidelobes) .  It  also  increases  n  and  thus 
decreases  Pc  for  MSI  data  even  more.  For  USI  data,  the  overlap  of  two  images  have  positive 
random  variables  with  a  positive  mean  probability  density  function  (rather  than  a  zero-mean 
probability  density  function  as  occurs  in  MSI  data).  Thus,  for  USI  data,  SNRg  is  propor¬ 
tional  to  /n  and  Pc  improves  as  n  increases,  whereas  for  MSI  data  SNRg  is  independent  of  n 

and  Pc  degrades  as  n  increases  (unless  other  preprocessing  operators  are  used).  Thus,  in¬ 
creasing  n  by  whitening  improves  the  performance  of  a  USI  correlator,  but  degrades  the  per¬ 
formance  of  an  MSI  correlator.  This  insight  is  useful  in  explaining  why  various  preproces¬ 
sing  operators  appear  to  be  useful  in  some  MSI  registration  applications,  but  not  in  others. 
In  some  MSI  registration  work,  the  image  was  dominated  by  one  key  object,  in  which  case  n  » 
1.  For  this  case,  the  conventional  correlation  applied  to  such  data  will  yield  satisfactory 
results  (simply  because  the  data  is  really  USI  not  MSI) . 

we  are  quite  concerned  with  the  fact  that  the  results  of  the  ad  hoc  applica- 


In  our  work, 


tion  of  different  preprocessing  operators  to  image  pairs  (with  no  attention  to  the  image 
models)  not  be  extended  to  the  general  case.  We  summarize  these  remarks  in  Table  1  and  re¬ 
emphasize  the  difference  between  US I  and  MSI  data.  Our  o  measure  in  (1)  is  useful  in  this 
regard.  For  pure  USI  data,  0  *  1;  whereas  for  pure  MSI  data,  o  =  0.  Vlhen  n  >>  1,  s  =  0  and 
conventional  correlation  is  useless.  When  SNRq  approaches  zero,  so  does  o.  However,  if  o 
is  large  (*  1),  SNRq  can  still  be  low  (e.g.  if  the  image  is  a  periodic  function).  Note  that 
0  need  only  be  evaluated  at  one  point  (the  correct  registration  point).  Thus,  we  can  calcu¬ 
late  p  for  a  few  representative  image  pairs  in  a  large  image  ensemble  and  use  the  preproces¬ 
sing  appropriate  for  that  o  value  for  the  entire  image  data  set. 


Table  1.  Dependence  of  USI  and  MSI  statistics  on  n  (Whitening  only) 
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In  image  registration,  the  correlation  length  n  of  the  data  is  another  parameter  useful 
in  analysis  (especially  concerning  spectrum  whitening).'  If  n  =  0,  an  image  (or  more  proper¬ 
ly  speaking,  its  pixels)  is  (are)  said  to  be  uncorrelated  and  its  spectrum  is  nearly  white. 
In  the  more  conroon  case,  n  M  0  and  often  n  *  1  and  then  the  image  (or  its  pixels)  is  (are) 
highly  correlated.  Much  work  on  image  enhancement  and  spectrum  whitening  exists  (4-5,  15- 
17).  However,  quite  different  cases  (no  noise  and  noise),  performance  measures  and  appli¬ 
cations  were  considered  in  these  works.  A  summary  of  this  work  is  thus  of  use  in  clarifying 
the  results  obtained  and  the  different  cases  being  addressed.  In  missile  guidance,  we  first 
distinguish  between  the  acquisition  phase  of  imagp  registration  (in  which  the  problem  is  to 
detect  the  image)  and  the  homing  phase  (in  which  .the  accurate  registration  or  location  of 
the  sensed  image  within  the  reference  image  is  of  concern).  Our  SNRq  in  (2)  is  appropriate 
for  the  first  case;  whereas  in  the  second  case  our  concern  is  to  minimize  the  variance  of 
the  correlation  at  the  peak  location  (in  this  case,  the  denominator  in  (2)  is  replaced  by 
the  variance  of  the  correlation  at  the  registration  peak  point).  In  this  second  case,  a 
correct  registration  is  assumed  to  have  been  determined  and  the  concern  within  the  image 
registration  processor  is  the  accurate  location  of  the  correct  registration  point. 

Our  immediate  concern  in  this  present  work  is  to  optimize  Pq  rather  than  to  optimize  reg¬ 
istration  accuracy.  However,  much  of  the  prior  work  on  both  problems  is  of  use.  For  the 
case  of  no  noise  and  when  n  a  0  (  a  white  image  spectrum),  optimization  of  the  SNRq in  (2) 
requires  the  use  of  only  a  simple  conventional  correlator  [15].  When  n  *  1,  optimization 
of  the  SNRq in  (2)  requires  the  application  of  a  second  derivative  preprocessing  operator  to 
one  of  the  images  prior  to  correlating  them.  It  has  also  been  shown  [5]  that  applying  a 
second  derivative  operator  to  one  image  is  equivalent  to  applying  a  first  derivative  opera¬ 
tor  to  both  images.  Since  the  work  in  (151  assumed  no  noise  in  the  data,  it  is  of  minimal 
concern  to  us,  since  simpler  techniques  3uch  as  simple  image  differencing  can  be  used  in 
such  cases  rather  than  correlations.  When  noise  is  present  in  the  images  and  whenthe  cor¬ 
rect  registration  point  has  already  been  determined, optimization  of  the  image  registration  accu¬ 
racy  can  be  considered.  In  this  case,  a  noise-whitening  filter  is  necessary  [4,17]  and  a 
matched  spatial  filter  system  results  in  which  both  inputs  are  first  whitened  with  respect 
to  the  noise.  Using  conventional  noise  models,  the  noise-whitening  is  shown  to  be  equivalent 
to  a  first  derivative  plus  a  mixed  derivative  operator.  Thus,  for  the  USI  case,  both  ap¬ 
proaches  are  comparable  and  result  in  the  need  for  a  derivative  preprocessing  step.  In  the 
noise  case,  this  image  differentiation  performs  the  necessary  spectrum  whitening  with  respect 
to  the  noise. 

Although  no  formal  analysis  of  the  case  of  detection  optimization  on  MSI  data  with  noise 
present  has  been  advanced,  we  can  use  this  prior  work  [15-17]  and  a  communication  analogy  to 
develop  the  optimum  processor  for  our  MSI  case.  In  this  case,  we  model  the  additive  noise  n 
as  the  sum  of  the  input  noise  plus  the  sensed  image  evaluated  far  from  the  correct  registra¬ 
tion  poinf.  We  find  that  the  reference  and  sensed  images  should  both  be  whitened  with  re¬ 
spect  to  n  prior  to  correlation.  A  more  important  observation  that  results  is  that  optimi¬ 
zation  of  the  SNR  at  the  correlation  peak  (for  .registration  accuracy  optimization)  is  not 
equivalent  to  optimization  of  our  detection  SNR  in  12)  unless  the  noise  spectrum  has  statis¬ 
tics  similar  to  the  imagery  itself.  In  the  cases  for  which  this  technique  has  been  used  [4, 
16,17],  this  has  been  the  case,  since  the  noise  in  the  imagery  was  defined  as  the  difference 
between  the  images.  In  this  case,  the  statistics  of  the  noise  n  in  the  imagery  happens  to 
agree  with  our  n  definition  by  implicit  assumption  and  thus  the  techniques  in  [4,16,17]  are 
appropriate.  However,  for  general  tWi  data,  the  noise  and  image  statistics  differ  and  our 
techniques  are  necessary  (i.e.  whitening  both  the  reference  and  sensed  images  with  respect 
to  n  and  applying  Gaussian  histogram  reshaping) . 


A  subtle  point  associated  with  our  communication  analogy  is  that  we  do  not  assume  the  a 
priori  use  of  a  correlator.  Rather,  we  derive  it  and  include  the  Gaussian  histogram  reshap¬ 
ing  outcome  of  that  derivation  in  our  architecture  of  Figure  1.  From  these  considerations 
and  by  proper  analysis  of  prior  work,  our  general  MS  registration  system  of  Figure  1  resul¬ 
ted.  From  our  assumptions  on  the  low  frequency  of  image  edges  in  our  model,  practical 
imagery  is  expected  to  have  low  n  and  thus  the  first  derivative  operator  is  expected  to  ap¬ 
proximate  an  image  whitening  operator.  We  thus  include  these  considerations  and  our  obser¬ 
vation  that  the  polarity  of  the  edges  in  a  dif f erentiated  image  should  be  equalized  by  a 
nonlinear  operator  such  as  the  absolute  value  operator  (for  MSI  data)  and  we  thus  consider 
the  use  of  the  ; GRAD |  operator  in  (3)  to  perform  the  three  operations  of  spectrum  shaping, 
edge-enhancement  and  polarity  equalization  shown  in  Figure  i. 

4.  MSI  DATA  BASES 


To  experimentally  verify  the  theory,  remarks  and  models  advanced  in  [13]  and  in  Sections 
2  and  3,  six  real  and  synthetic  MS  image  data  bases  were  used.  Two  sets  of  color  images  of 
the  same  scene  in  the  blue,  green  and  red  primary  color  bands  were  used  as  simulated  MS  data. 
Our  results  indicated  that  such  data  were  nearly  pure  OSI.  Any  preprocessing  was  found  to 
yield  poorer  results  than  a  conventional  correlation  (as  predicted  from  Section  3) .  Any 
threshold  operator  (with  any  choice  of  the  threshold)  also  gave  poorer  results  (due  to  the 
high  SNRi  of  the  data)  since  this  operator  actually  removed  low  intensity  image  data  rather 
than  noise.  This  is  also  in  agreement  with  the  theory  in  [13)  which  indicated  that  for 

large  SNRj  data,  the  optimum  threshold  was  zero.  Thus,  no  experimental  results  are  included 

on  this  data,  although  the  results  obtained  clearly  affected  our  analysis  and  selection  of 
future  MS  image  data  bases. 

vie  used  real  downlooking  planimetric  imagery  of  Ames  Research  Center  (MS  data  base  3) 
taken  in  three  MS  bands  (Table  2)  to  analyze  the  effects  of  p  and  SNR^  on  the  need  for  the 
I  GRAD  I  »  GRD  preprocessing  operator.  Our  results  are  presented  in  Section  S.  We  refer  to 
this  data  base  as  downlooking  target  (DLT)  imagery.  In  Tables  3  and  4,  we  summarize  the 
contents  of  our  MS  image  data  bases  4  and  5.  We  refer  to  these  as  short  range  targetlooking 
(STL)  and  long  range  targetlooking  (LTL)  respectively.  They  contain  images  of  the  same  Ames 
site  as  in  Table  2  but  at  a  depression  angle  of  30*  and  with  a  slant  range  of  2880  feet  and 
an  altitude  of  1440  feet  (STL  data  of  Table  3)  and  with  a  longer  slant  range  of  5000  feet 
and  an  altitude  of  2550  feet  (LTL  data  of  Table  4)  corresponding  to  the  same  30*  depression 
angle.  The  major  difference  in  these  data  are  the  larger  number  n  of  objects  present  in  the 

LTL  data  compared  to  the  STL  data.  In  Section  6,  we  analyze  how  the  number  n  of  objects  in 

the  image  affects  the  SNRq  obtained.  The  effects  of  Gaussian  histogram  reshaping  on  the 
correlation  of  these  data  are  addressed  in  Section  7  and  verification  of  the  MSI  model  ad¬ 
vanced  in  [13]  and  summarized  in  Section  2  is  provided  in  Section  8  for  these  data. 


Table  2.  Downlooking  target  (DLT)  Ames  data  base 
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Table  3.  Short  range  (2880  feet)  target¬ 
looking  (30*)  (STL)  Ames  data  base 
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Table  4.  Long  range  (5500  feet)  target¬ 
looking  (30*)  (LTL)  Ames  data  base 


IMAGE 

SENSOR 

X(ynt) 

$22 

Thermal  IR 

00 

1 

a— 

«\> 

S23 

Visible 

0.38-0.73 

S  24 

Near  IR 

1.06 

S  25 

Near  IR 

0.8-0.95 

A  simulated  MSI  data  set  (denoted  by  SMS)  was  also  prepared  by  generating  and  randomly 
adding  circles  of  different  radii,  central  locations  and  gray  levels.  The  SMS  image  data 
pairs  so  generated  represent  pure  MSI  data.  In  Section  9,  we  compare  the  use  of  high  pass 
filtering  (HPF)  and  the  |GRADj  operators  to  the  whitening  (WIT)  operator  for  this  synthetic 
data  as  well  as  for  the  LTL  data  set.  Our  primary  use  of  the  SMS  data  is  in  verification  of 


the  optimal  threshold  level  to  use  (see  Figure  1  and  [13))  as  a  function  of  the  edge  pulse 
rate  X  and  the  SNRj_of  the  input  data.  In  Section  10,  we  analyze  this  issue  by  adding  con¬ 
trolled  amounts  of  white  Gaussian  noise  to  the  SMS  data,  performing  the  indicated  preproces¬ 
sing  and  comparing  the  results  for  various  threshold  and  SNR^  levels.  Our  summary  and  con¬ 
clusions  are  then  advanced  in  Section  11. 

As  indicated  in  Tables  2-4,  we  denote  the  different  image  pairs  used  in  all  digital  cor¬ 
relation  simulations  by  specific  numbers  S2  -  S25.  The  simulated  imagery  are  referred  to  as 
SIM1  -  SIM4,  where  X  =  14  objects  per  image  length  are  used  for  SIM1  and  SIM2  data  and  X  -  10 
for  SIM3  and  SIM4  data.  We  denote  the  preprocessing  operator  used  by  the  abbreviations: 

SGN  (gradient  with  sign  preserved),  GRD  (absolute  value  of  the  gradient  as  defined  in  (3)), 
GAS  (absolute  value  of  the  gradient  with  Gaussian  histogram  equilization  applied)  following 
the  image  number.  We  denote  the  different  high  pass  filtered  versions  of  each  image  by  FILL 
FIL2,  etc.  and  the  absolute  values  of  these  images  by  AFIL1,  AFIL3,  etc.  (see  Section  9). 

Our  experimental  data  generally  contain  experiment  number,  input  image  pairs  correlated,  the 
output  correlation  SNRg value  obtained,  the  measured  o  value  for  the  image  pair  and  remarks 
on  the  experiment  as  necessary. 


5.  0  MEASURE  AND  j  GRAD  j  OPERATOR 

In  Table  5,  we  list  our  experimental  data  obtained  on  the  DLT  data  base  of  Table  2.  Ex¬ 
periment  one  shows  the  large  SNRq  *  21.3  autocorrelation  obtained.  Comparing  correlation 
patterns  from  experiments  one  ana  two  (Figure  2)  shows  that  the  !GRADj  preprocessing  opera¬ 
tor  yields  a  larger  SNR.  value  and  a  narrower  correlation  peak  width  (due  to  the  spectrum 
whitening  produced  by  the  edge-enhancement  operator) .  Comparing  the  results  of  the  cross¬ 
correlation  experiments  three  and  four  (Figure  3),  we  see  that  for  a  low  (o  «  0.26)  corre¬ 
lation  coefficient  value  (as  occurs  here),  SNR0  is  remarkably  improved  by  the  i GRAD |  =*  GRD 
preprocessing  operator.  From  Table  5  (see  *  note)  and  Figure  3,  we  also  note  that  the 
registration  accuracy  of  the  correlation  on  GRD  preprocessed  data  is  better.  Similar  re¬ 
sults  are  obtained  (experiments  five  and  six)  for  image  pairs  eight  and  ten  (because  of 
their  low  p  «  0.28  correlation  coefficient).  However,  for  image  pair  ten  and  eleven  (exper¬ 
iments  seven  and  eight),  their  p  «  0.81  value  was  quite  large.  This  is  due  to  the  common 
polarity  of  the  edge  data  in  this  image  pair.  This  image  pair  thus  corresponds  to  USI  rather 
than  MSI  data.  The  observed  4.8  loss  in  SNRq  of  the  GRD  preprocessed  imagery  for  this  pair 
verifies  the  theory  advanced  in  [13]  and  Sections  2  and  3.  From  these  experimental  data,  we 
conclude  that  p  is  a  good  measure  of  whether  a  given  image  pair  is  USI  or  MSI  (with  o  •  0.5 
being  an  approximate  break  point) .  We  also  conclude  that  GRD  preprocessing  improves  the 
correlation  SNRq  of  MSI  imagery  and  degrades  that  of  USI  imagery  and  that  better  registra¬ 
tion  accuracy  and  narrower  correlation  peaks  result  from  the  use  of  GRD  data  and  thus  that 
spectrum  whitening  is  performed  by  the  GRD  operator.  More  experimental  data  than  the  small 
sampling  given  in  Table  5  was  obtained  and  used  to  support  our  conclusions.  However,  space 
permits  the  inclusion  of  only  selected  but  representative  examples  of  these  results. 

6.  EFFECT  OF  THE  NUMBER  n  OF  OBJECTS 


In  Table  6,  we  summarize  the  results  of  correlations  obtained  on  the  STL  and  LTL  image 
data  bases  with  attention  to  the  SNRq  improvement  obtained  using  the  GRD  operator  as  a  func¬ 
tion  of  the  number  of  objects  n  in  the  input  imagery.  In  experiments  one  and  two,  we  show 
the  autocorrelation  SNR  gain  (9.8  vs  21.2)  obtained  due  to  the  larger  number  of  objects  n 
present  in  the  LTL  data* .  In  experiments  three-four  and  five-six,  an  SNR0  improvement  of 
2.3  vs  3.1  and  2.9  v?  6.4  were  obtained  with  the  GRD  preprocessing  operator.  This  corre¬ 
sponds  to  an  average  SNRq  gain  by  a  factor  of  1.8  for  the  STL  data.  In  experiments  seven- 
eight  and  nine-ten,  we  consider  similar  data  for  the  long  range  imagery  (LTL)  containing 
more  objects.  The  SNRg  improvements  (3.3  vs.  11.6  and  4.6  vs  14.4)  represent  a  much  larger, 
gain  factor  of  3.4.  From  these  data,  we  conclude  that  the  use  of  the  | GRAD i  »  GRD  prepro¬ 
cessing  operator  becomes  more  necessary  and  improves  the  performance  obtained  by  a  larger 
factor  as  the  number  n  of  objects  in  the  image  increases  (and  henoeas  the  imagery  more  closely 
approaches  that  of  our  pure  MSI  model) .  As  before,  additional  data  beyond  that  included  in 
Table  6  is  available  to  substantiate  these  conclusions.  In  addition,  the  image  registration 
accuracy  is  better  and  the  width  of  the  correlation  peak  is  narrower  when  GRD  preproctssed 
data  is  used.  These  trends  verify  our  theoretical  remarks  advanced  in  [13]  and  Section  3. 


Larger  n  also  means  large  Signal  Bandwidth  Product  (SBWP) 


S8.GRD  ®  s8.GRD 
SNRg  *  35.2 


Figure  2  Demons tra t i on  of  the  whitening  effect,  in¬ 
creased  SNRg  and  narrower  correlation  oeak 
width  obtained  by  the  GRD  =  | GRAD  |  ooerator. 
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figure  3  Low  o  *  0.26  data  implies  poor  registration 

accuracy  and  low  SNRg.  'GRADl  *  GRD  increases 
SNRg,  whitens  image  soectrum  (narrower  peak) 
and  gives  better  registration  accuracy. 


Table  5.  Correlation  data  for  the  DLT  data  base 


EXPERIMENT 

INPUT-REF 

Ti 

SNR0 

COMMENTS 

1 

S8ft58 

1.0 

21.3 

Auto  co rre 1  a t i on 

2 

S3.GRD$S8.GRD 

1  .0 

35.2 

Auto  (.‘.USI),  GRD  does  whitening 

3* 

S8«SU 

0.26 

3.0  " 

tc  is  small,  thus  MSI;  GRD 

4 

S8.GRD0S1  1  .GRD 

0.36 

11.7. 

(improves  SNRo 

5* 

j  S8CS 1 0 

0.28 

2.9  - 

1 p  is  small,  thus  MSI;  GRD 

6 

!  S8.GR0gS10.GR0 

0.22 

7.0  . 

(improves  SNRq 

7 

;  s  i  o0s  n 

0.81 

1  6  .  9— 

Ip  is  large,  thus  USI;  GRD 

8 

Sl0.GRCgS11.GRQ 

0.48 

12.1  . 

(degrades  SNRq 

•Peak  location 

in  error  by  3  pixels 

Table  6.  Effect  of  number  n 

of  Objects  on  MSI  Correlations 

EXPERIMENT 

INPUT-REF 

0 

SNRQ 

COMMENTS  1 

I  STL 

S2®$  2 

1.0 

9.8 

More  objects,  more  SBWP  and 

2  LTL 

$2  20S  22 

1.0 

21.2 

larger  autocorrel  SNRQ 

3  STL 

S20S4 

0.23 

2.3 

4  STL 

S2  .  GRD  ©S4 . GRD 

0.08 

3.1 

Average  |  GRAD |  improvement 

5  STL 

S2»S5 

0.33 

2.9 

is  1.8 

6  STL 

S2.GRD&S5.GRD 

0.24 

6.4 

7  LTL 

S22®  S2  3 

0.37 

3.3 

8  LTL 

S22  .  GRD®  S2 3  .  GRD 

0.26 

11.6 

Average  ) GRAD  1  improvement 

9  LTL 

S22»S25 

0.27 

4.6 

is  3.4 

10  LTL 

S22.GRD&S25.GR0 

0.29 

12.5 

7.  NEED  FOR  GAUSSIAN  HISTOGRAM  RESHAPING 

We  now  consider  the  LTL  image  data  base  with  attention  to  the  improvement  obtained  by  use 
of  the  GAS  preprocessing  operator  (including  Gaussian  histogram  reshaping)  compared  to  the 
GRD  (magnitude  of  the  gradient)  preprocessing  operator  alone.  Comparing  the  results  of  ex¬ 
periments  one-two  in  Table  7,  we  find  a  gain  of  6%  for  the  autocorrelation  case.  From  ex¬ 
periments  three-five  a  13%  gain  is  observed  due  to  the  GAS  vs  the  GRD  preprocessing  operator 
From  experiments  six-eight,  a  larger  32%  gain  is  found.  From  these  brief  experiments  and 
others,  we  conclude  that  the  addition  of  a  Gaussian  histogram  reshaping  preprocessing  opera¬ 
tor  can  provide  a  15%  average  improvement  in  SNRq.  Analogous  data  taken  on  our  SMS  imaoerv 
agree  with  these  results. 

Table  7.  Improvement  obtained  (LTl)  by  Gaussian  Histogram  Shapening 


experiment 

INPUT-REF 

0 

SNR 

COMMENTS 

1 

S22.GRO0S22.GRD 

1 

U 

50.3 

2 

S22  ,  GAC0S22  .  GAS 

1 

52.9 

6%  Gain 

3 

S22  ®S23 

0.37 

3.3 

4 

S22.GRD8S23.GRD 

0.26 

11.6 

13%  Gain 

5 

SZ2  .  GAS0S23  .  GAS 

0.32 

13.1 

6 

S220S25 

0.27 

4.6 

7 

S22.GRD0S25.GRD 

0.29 

12.5 

32%  Gain 

8 

S22.GAS0S25.GAS 

0 . 36 

16.5 

8.  VERIFICATION 

OF  THE 

MS  MODEL 

In  Figure  4,  we  show  the  cross-sectional  scans  through  column  71  of  images  S22  and  S25 
after  GRD  preprocessing.  Close  scrutiny  of  the  data  is  necessary  to  idertify  common  edges 
in  the  two  scans.  To  facilitate  such  an  analysis,  eighteen  of  the  common  edges  in  these  two 
images  have  been  labeled  in  Figure  4.  From  this  figure,  we  note  that  the  amplitude  of  the 


common  edges  differs  quite  significantly  between  the  two  images  and  that  all  edges  are  net 
ideal  delta  functions  (but  rather  that  their  widths  depend  upon  the  sharpness  of  the  edge 
transitions  in  the  original  data) .  From  an  analysis  of  these  and  other  cross-sectional 
image  scans,  the  validity  of  our  MS  image  model  advanced  in  (13]  and  summarized  in  Section  1 
has  been  verified.  For  our  SMS  data,  nearly  exact  1  values  (low-density  Poisson  data  rates 
per  unit  image  length)  of  =  10  and  14  were  obtained,  measured  and  experimentally  verified 
(see  Sections  9  and  10) . 


0  20.0  40.0  50.0  80.0  100.0  120.0 

(4a)  (4b)  U8.0 


Figure  4  Cross-section*!  scans  through  column  71  of 
Images  S22.GRD  (left)  and  S25.GRD  (right) 
verifying  our  MSI  model. 


91  COMPARISON  OF  HPF,  GRP  AND  WIT  OPERATORS 

The  GRAD |  preprocessing  operator  appears  attractive  for  edge-enhancement  and  whitening 
of  MS  data,  where  polarization  equalization  is  necessary.  A  high  pass  filter  (HPF)  is  anoth¬ 
er  attractive  edge-enhancement  operator,  since  it  can  be  realized  optically  and  is  inherently 
present  in  the  synthesis  of  a  matched  spatial  filter  [181 .  It  is  straightforward  to  show 
that  the  width  of  the  pulses  in  the  train  of  pulses  produced  by  the  HPF  version  of  an  image 
decreases  as  the  cutoff  frequency  fg  of  the  KPF  is  increased  and  that  the  energy  in  the 
pulses  and  hence  the  useable  data  modulation  also  decreases  as  fg  increases.  Conversely, 
the  area  of  the  pulses  produced  by  the  GRD  operator  is  constant.  We  thus  expect  the  GRD 
operator  to  be  superior  to  the  HPF  edge-enhancement  operator. 

In  Table  8,  we  summarize  the  results  obtained  using  the  CRD,  HPF,  and  HPF  operators  for 
the  LTL  data  base.  To  properly  compare  these  operators,  various  fg  HPF  cutoff  frequencies 
were  tried  and  only  the  ones  with  the  best  SNRg  are  reported.  The  HP  filters  are  denoted 
by  the  suffix  FIL  (filter)  following  the  image  number.  The  results  of  correlations  on  data 
preproce3sed  using  the  absolute  value  of  the  HPF  operator  are  denoted  by  the  suffix  AFI2 
(the  best  filter  cutoff  frequency  was  again  used  in  these  experiments).  The  results  m 
Table  8  are  easily  explained.  First,  we  note  that  the  HPF  preprocessed  data  (experiments 
two  and  seven)  yield  better  SNRo  than  was  obtained  with  no  preprocessing  (experiments  one 
and  six).  This  indicates  that  the  image  pairs  are  not  pure  MSI,  since  if  they  were,  SNRg  = 

0  would  have  been  obtained  (see  Section  3).  The  absolute  value  of  the  HPF  operator  shows  a 
loss  in  SNRg  in  both  cases.  This  occurs  because  the  polarity  of  the  edges  in  the  image  pairs 
contain  common  information  (again,  this  occurs  because  the  data  are  not  pure  MSI).  The  GRD 
operator  yields  larger  SNRg  values  than  any  HPF  operator  and  the  GAS  operator  (the  absolute 
value  of  the  GRAD  plus  Gaussian-histogram  reshaping)  yields  still  better  results. 

From  these  data,  we  conclude  that  the  HPF  operator  is  inferior  to  the  GRD  one,  even  for 
non-pure  MSI  data.  In  Figure  5,  we  show  three  HP  filtered  versions  of  image  S22  with  in¬ 
creasing  cut-off  frequency  Eg.  As  predicted,  the  edges  of  the  image  become  more  enhanced  as 
f q  is  increased,  but  the  image  contrast  decreases  (i.e.,  the  energy  in  the  edaes  decreases). 
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Table  8.  Edge-enhancement  operator  tests  (LTl  data; 


experiment 


INPUT-REF 


1  1 

52  2(2 

£2  3 

3.3 

i  1  1 

S22.FH.2lJ 

j?S23.FIL2 

9.7 

S22.AFIl2®S23.AFIL2  8.0 


S22.GRDffS23.GRD 
S22  .GAS®52  3  .GAS 


COMMENTS 


9.7  HPF  better  t nan  no  preproc 
(not  MSI ) 

8.0  Abs  Value  gives  loss  since 
not  MSI 

11.6  GRD  better  than  HPF 

13.1  GAS  bes  t  of  all 


HPF  better  than  no  preproc 
(not  MSI) 

Abs  Value  gives  loss  since 
not  MSI 

GRD  better  than  HPF 
GAS  best  of  all 


m.nu 


(c)  iii. nu 


Figure  5  Three  filtered  versions  of  image  S22  with  increasing  cutoff  frequency. 


Similar  experiments  were  performed  on  the  SMS  imagery  with  even  more  dramatic  results 

synthetlc  data  «as  essentially  pure  MSI) .  The  highlights  are  included  in  Table 
;,/^eXperlroentS  °ne  an<?  tW°'  we  366  that  negligible  SNR0  occurs  for  the  cross-correla- 
S  PrePJoce®sln9  (as  is  expected  for  MS  data).  A  WIT  (whitening)  operator  was 

three?P  1SinCe°thil«Snftoa‘  1  9*V?  exceUent  results  for  the  auto-correlation  (experiment 
three).  Since  this  case  corresponds  to  pure  USI  data,  this  result  is  expected.  The  much 

thatrwhitLinoal«ed  thC  WIT  °Perat0r  in  cross-correlations  (experiment  four)  verifies 

=9  et ,°f  jSe  Pure  MSI  data-  This  experiment  four  also  verifies  that  the 
image  pair  is  uncorrelated  and  thus  represents  a  pure  MSI  pair .  From  experiments  five  and  six, 
we  see  that  the  HPF  operator  without  the  absolute  value  operator  is  of  little  use  on  cure  MSI 

e*P®J;:unent  seven,  we  find  that  our  special  GAS  operator  provides  much  better  SNR 
than  does  any  other  preprocessing  operator. 


EXPERIMENT 


Table  9.  Edge-enhancement  operator  tests  (SMS  data) 


INPUT-REF 

SNR 

1  SIM1#SIM1 

6.3 

SIM1«SIM2 

0.6 

SIM1.WITfCSIMl.WIT 

183 

SIM1.WIT«SIM2.WIT 

3.7 

SIMl.FIL3ffSIM2.FIL3 

2.3 

S!Ml.AFIL3felM2.AFIL3 

15.8 

SIM1.GAS®SIM2.GAS 

32.7 

COMMENTS 


Low,  thus  nearly  pure  *1SI 
WIT  helps  pure  USI  (=  auto-correl) 
WIT  not  good  for  nearly  pure  MSI 
) HPF  without  abs. value 
)  useless  for  nearly  pure  MSI 
GAS  excellent  for  nearly  pure  MSI 


10.  THRESHOLD  SELECTION  AS  A  FUNCTION  OF  SNRi 


S~/ 

T-. 

s’ 


In  the  major  experiments  performed  on  a  pair  of  SMS  data,  zero-mean  white  Gaussian  noise 
was  added  to  one  of  the  images  to  produce  two  SMS  images  with  SNRi  «  3  and  SNRi  «  4.  Thres 
holded  versions  of  these  images  with  different  thresholds  were  correlated  with  a  GAS  prepro¬ 
cessed  version  of  the  other  image  (with  different  gray  level  distributions) .  The  resultant 
degradation  SNR/SNRo  in  SNRo  from  the  SNRi  «  »  case  (SNRo)is  plotted  in  Figure  6  versus  the 
normalized  threshold  Th  for  the  two  SNRi  cases.  Although  these  data  were  obtained  for  one 
isolated  image  correlation  pair  rather  than  the  statistical  ensemble  necessary,  the  results 
agree  quite  well  with  those  predicted  by  theory  [13],  A  clear  peak  in  D  «  SNR/SNRo  is  ob¬ 
served  for  Th  ^  0,  the  optimum  Th  increases  as  SNRi  decreases,  and  the  degradation  becomes 
worse  as  SNRi  decreases.  These  experimental  data  suggest  that  our  optimum  Th  parameter 
selection  technique  [13]  is  valid. 


NORMALIZED  THRESHOLD 


Figure  6  Experimentally  determined  degradation  in  SNRo  from  SNRi  *  -  case 
(SNRo)8**  function  of  the  normalized  threshold  for  two  different 
SNR^  values  using  SMS  data. 


11.  SUMMARY  AND  CONCLUSIONS 

In  this  paper,  a  new  multisensor  image  model  using  random  variables  to  represent  the  am¬ 
plitude  and  location  of  the  edges  in  an  image  was  reviewed  and  experimentally  verified.  A 
general  multisensor  image  pattern  recognition  processing  architecture  with  a  wide  variety  of 
preprocessing  operators  was  also  reviewed  and  the  need  for  the  various  preprocessing  opera¬ 
tors  was  analyzed  by  experiments.  The  nonlinear  polarization  equalization  operator  and  the 
Gaussian  histogram  reshaping  operator  had  not  been  included  in  prior  multisensor  processors. 

Our  experiments  showed  that  the  correlation  coefficient  o  was  a  useful  measure  of  whether  an 
image  pair  was  USI  or  MSI  with  o  >  0.5-0. 6  being  indicative  of  USI  data. 

For  MSI  data,  we  showed  that  HPF  was  inferior  to  the  GRD  operator,  that  HPF  without  an 
absolute  value  operator  was  useless,  that  Gaussian  histogram  reshaping  improved  the  output 


wans 


correlation  by  an  average  of  15%.  We  verified  the  major  theoretical  remarks  we  advanced 
earlier  [13)  regarding  how  the  number  of  independent  objects  in  the  image  affects  the  data 
and  moreso  that  our  theory  by  which  the  optimal  threshold  can  be  selected  was  valid.  These 
experimental  verifications  provided  much  insight  into  the  nature  of  multisensor  image  pattern 
recognition. 
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ABSTRACT 

A  hyperspace  description  of  a  matched  spatial  filter  (MSF)  correlator  is  advanced  and  used 
to  develop  a  new  MSF  that  is  a  synthetic  discriminant  function  (SDF) .  The  use  of  this  fil¬ 
ter  to  recognize  objects  independent  of  their  aspect  is  described.  Terminal  homing  recogni¬ 
tion  of  tanks  from  IR  imagery  is  taken  as  a  case  study.  Bandpass  filtering  and  a  new  maxi¬ 
mum  common  information  filter  synthesis  concept  and  issues  such  as  filter  energy  are  described 
and  used  to  improve  the  system's  intra-class  pattern  recognition  ability  and  to  overcome  the 
intensity  differences  present  in  IR  imagery.  Experimental  verification  is  also  included. 

1.  INTRODUCTION 

The  recognition  of  the  presence  of  a  reference  object  in  a  cluttered  background  and  the 
location  of  its  position  are  old  and  familiar  problems  in  pattern  recognition  [1].  Correla¬ 
tion  of  the  on-line  input  image  with  a  stored  reference  pattern  is  one  of  the  more  attrac¬ 
tive  solutions  to  such  problems.  This  operation  can  be  realized  both  optically  (in  a  fre¬ 
quency  plane  correlator  or  in  other  architectures  [2])  or  digitally  (using  FFT  algorithms 
[3]).  However,  in  most  practical  pattern  recognition  problems,  the  input  image  and  the  ref¬ 
erence  function  will  differ  due  to  intensity  differences  (this  is  especially  prevalent  when 
multisensor  or  IR  imagery  is  used)  and  moreso  due  to  geometrical  differences  (this  problem 
is  of  major  concern  when  the  aspect  angle  from  which  the  object  is  viewed  cannot  be  con¬ 
trolled)  .  Our  specific  pattern  recognition  case  study  concerns  the  recognition  and  location 
of  M-60  tank  targets  from  IR  imagery.  In  this  application,  both  geometrical  and  intensity 
differences  exist  between  the  input  and  reference  data.  . 

Because  of  the  ease  with  which  a  correlation  can  be  performed  optically,  we  consider  the 
use  of  a  frequency  plane  correlator  for  this  pattern  recognition  problem.  However,  since 
the  performance  of  a  correlator  is  well-known  to  seriously  degrade  when  random  contrast  re¬ 
versals  [4]  or  geometrical  distortions  [5]  exist,  we  suggest  a  hyperspace  description  of  the 
data,  the  MSF  and  the  correlator  (Section  2)  to  overcome  these  problems.  Using  this  hyper¬ 
space  description,  we  develop  a  synthetic  discriminant  function  (SDF)  and  use  it  as  a  syn¬ 
thetic  MSF  in  our  optical  correlator.  In  Section  3,  we  review  our  prior  [6,7]  SDF  approach 
in  terms  of  this  new  hyperspace  formulation.  In  Section  4,  we  review  three  techniques 
(Fourier  transform  kernels,  Gram-Schmidt  (G-S)  techniques  and  Karhunen-Loeve  (K-L)  tech¬ 
niques)  for  basis  function  selection  with  emphasis  on  our  new  G-S  method.  When  additive 
noise  of  zero-mean  is  present,  modifications  to  the  SDF  synthesis  procedure  are  required. 
These  issues,  derivation  of  the  correlation  SNRq  in  terms  of  hyperspace  parameters  and  new 
concepts  such  as  maximum  filter  energy  and  the  maximum  common  information  (MCI)  filter  are 
then  developed  in  Section  5.  Experimental  verification  and  demonstration  of  this  SDF  ap¬ 
proach  to  optical  MSF  pattern  recognition  are  provided  in  Section  6.  Our  present  attention 
lies  with  improving  the  intra-class  pattern  recognition  performance  of  the  system  (i.e.  the 
ability  of  the  SDF  to  recognize  different  aspect  views  of  the  same  reference  object  as  mem¬ 
bers  of  the  same  class).  In  a  companion  paper  [81,  we  consider  extensions  of  these  tech¬ 
niques  for  inter-class  discrimination  of  tanks  from  other  false  targets  and  from  non  zero- 
mean  background  noise. 


2.  HYPERSPACE  DESCRIPTION  OF  A  MSF  CORRELATOR 


To  provide  a  new  filter  function  for  use  in  a  correlator  that  is  capable  of  maintaining  rec¬ 
ognition  of  a  reference  object  independent  of  geometrical  distortions,  we  use  an  alternate 
hyperspace  description  of  the  input  data  and  the  filter  function.  We  use  1-D  functions  for 
simplicity.  Each  of  the  members  of  the  input  data  set  (fn(x)}  of  objects  to  be  recognized 
is  written  as  an  expansion  in  terms  of  the  basis  functions  4>n , ( x)  and  weightino  coefficients 
^  as 


f  (x)  =  I  a  4>  (x)  .  (1) 
m  m  m 

Each  f  in  . fn;  is  plotted  as  a  M-dimensional  vector  in  an  M-dimensi onal  hyperspace  with  axes 
sm  and  with  the  projections  on  each  axis  being  the  a,,,  coefficients  in  (1).  Thus,  each  input 
can  be  described  as  a  vector 
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The  basis  function  set  is  orthonormal  and  different  coefficients  exist  for  the  N 

different  fn  members  of  tne  set  ifn;. 

Two  surfaces  in  hyper  space  that  are  quit.eAuseful  are  the  hypersphere  and  the  hyperplane. 

A  hypersphere  of  radius  Rs  is  described  by  f  •  f  =  Rs(a  constant).  This  corresponds  to  an 
equi-energy  surface.  This  can  easily  be  seen  from  the  definition  of  the  energy  ej  of  the 
function  f  as 

ef  =  .'f2{X)  dx  =  ^  a2  =  f  ■  f  =  Rg.  (3) 

A  hyperplane  also  has  physical  significance  as  the  product  or  two  vectors  (one  of  which  g  is 
fixed) .  It  also  corresponds  to  the  correlation  Rf„  output  of  two  functions  f  and  g  (evalua¬ 
ted  at  t  =  0),  i.e. 

£fg  *  1  •  *  =  lambm  =  ^  =  f©g!t=0  =  Rfg(0)  =  V  (4) 

In  (4)  ,  we  assume  that  both  f  and  g  are  expanded  in  terms  of  the  same  1 4>m '  basis  set  and  that 
they  have  coefficence  am  and  respectively.  The  energy  eE-  in  the  output  correlation,  with 
no  shift  (t  =  0)  between  f  and  g,  is  Rf  (0.)  .  If  g  is  fixed  y(i.e.  if  the  bm  are  fixed),  then 
the  ?  •  g  =  Rp(a  constant)  surface  is  a’hyperplane .  This  describes  the  output  of  a  correla¬ 
tor  (f0g)  with  the  requirement  that  the  correlation  equal  the  constant  Rp  value. 


FIGURE  1,  Simplified  (2-axis)  hyperspace  pattern  recognition  problem 

In  our  MSF  correlation  analogy,  the  fixed  reference  function  g  is  our  MSF  h.  It  corre¬ 
sponds  to  the  vector  drawn  from  the  origin  in  hyperspace  normal  to  the  f  •  g  =  Rp  hyperplane. 
Thi3  hyperplane  describes  the  case  of  a  constant  output  from  the  correlator.  Any  f  in  (fn) 
when  correlated  with  g,  should  lie  on  this  hyperplane  to  be  recognized.  In  Figure  1,  we 
show  a  simple  pattern  recognition  example  for  a  M  »  2-axis  hyperspace.  The  X’s  are  one 
class  of  objects  to  be  recognized  and  the  0’s  are  another  class  of  objects  to  be  separated. 
Surface  1  i;i  Figure  1  describes  a  hypersphere  with  Rg  =  Rgl.  Surface  2  describes  a  hyper¬ 
plane  corresponding  to  Rp  «  [g[  =  Rpi*  Surface  3  represents  a  more  complex  hypersurface. 

To  recognize  the  X's  and  separate  them  from  the  object  class  0,  we  can  require  f  •  g  <_  Rpy 
(surface  2)  or  ?  •  ?  =  Ef  <_  Rsi.  Our  major  concern  in  this  paper  is  how  to  select  the  { 5m } 
to  produce  the  excellent  separation  of  the  different  object  classes  (inter-class  pattern 
recognition)  and  the  excellent  clustering  of  the  data  for  each  class  (intra-class  pattern 
recognition)  shown  in  this  simplified  example  in  Figure  1.  The  presence  of  noise  in  the  data 
will  appear  centered  at  the  origin  in  hyperspace  as  well  as  around  the  hyperplane.  In  these 
cases  more  advanced  techniques  (Sections  3-5)  are  required. 

3.  SYNTHETIC  DISCRIMINANT  FUNCTION  (SDF!  SYNTHESIS 


In  this  section,  we  describe  our  general  synthetic  discriminant  function  synthesis  tech¬ 
nique  and  how  it  is  used  within  our  hyperspace  description  of  a  MSF  pattern  recognition  cor¬ 
relator.  We  write  the  ?n  members  of  our  training  set  {fnl  of  input  objects  to  be  recognized 
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f  =  -  a  v  . 
n  m  run  m 


We  denote  our  desired  filter  function  by  h.  We  write  it  as  a  similar  expansion  in  the  same 
\ 5_;  basis  set  as 

h  =  I  b  3  •  ( 6 ) 

in  n  m 

We  require  the  correlation  of  all  fn  with  h  to  equal  a  constant  R  .  When  the  are  ortho¬ 
normal,  this  condition  reduces  to  p 


fn  '  h 


R,  v.(0)  =  fa  b_  =  R  =  constant, 
r^n  m  ruft  ni  p 


Our  pattern  recognition  problem  can  now  be  stated  quite  simply.  We  desire  to  find  the 
filter  function  fi  in  (6)  that  satifies  (7)  for  all  f  in  the  training  set  ■ fp  ■ .  To  do  this, 
we  must  determine  the  bm  and  the  Jm.  Four  steps  are  required  to  achieve  this.  We  first 
form  the  autocorrelation  matrix 

Rf  f  (0)  =  £i  •  $y  (8) 

of  all  fn  inputs.  We  then  diagonallize^the  R  matrix  in  (8).  Techniques  to  do  this  are  de¬ 
scribed  in  Section  4.  This  yields  the  We  then  determine  the  a^  coefficients  in  (5) 

from 


To  find  the  we  pick  the  constant  Rp  in  (7)  (in  this  paper  we  arbitrarily  select  RD  =  1 ) 
and  use  the^a^,,  found  from  (9)  in  (7)  .  We  then  know  the  and  the  bn,  and  hence  can  form 
the  filter  h  in  (6)  with  the  necessary  correlation  feature  in  (7) . 

4.  BASIS  FUNCTION  SELECTION 

Let  us  now  elaborate  on  the  various  techniques  by  which  the  (»-,}  basis  function  set  can 
be  determined  from  the  autocorrelation  matrix  R  in  (8) .  One  of  the  simplest  orthonormal 
basis  function  sets  are  the  Fourier  kernel  functions.  Duvernoy  (9-12]  and  Caulfield  [13-14] 
have  used  these  in  analogous  SDF  synthesis.  However,  details  of  the  steps  involved  in  their 
filter  synthesis  and  how  the  Fourier  coefficients  to  be  retained  were  chosen  have  not  been 
provided.  Moreso,  they  selected  quite  specific  pattern  recognition  problems  for  which  the 
Fourier  coefficients  appear  to  represent  an  attractive  basis  set  for  data  discrimination. 

Our  concern  is  to  develop  a  more  general  and  organized  approach  to  basis  function  selection 
rather  than  relying  on  ad  hoc  techniques  and  use  of  a  fixed  (Fourier  kernel)  basis  set  that 
might  not  always  be  best  for  a  given  general  pattern  -  ignition  problem. 

In  [6,71,  we  developed  a  new  G-S  technique  for  generation  of  a  general  basis  function  set. 
The  algorithm  is  described  by 

=  f1(x)/E1  ) 

<P2  ~  [f2(x)  "  Z2  <r21)  <x)  ^/E2 

m-1  >  <1°> 

*m  *  *  fSlZm(cmi>Vx)1/Em  f 

*M  “  [fMlX>  “  f  =  i  ltmf '  *1 lx)  1  /EM  _ ) 


where  R^ .  =  f ^  •  f ^  are  the  elements  of  the  autocorrelation  matrix  R^  .  . 

*  m  -s  *-3 

t_  =  Z  d  f  can  be  used  to  describe  the  m-th  basis  function  as  a  function  of  the 
n=l  m-th  input  fm  and  the  prior  f^  inputs  (£  <_  m  -  1)  . 

m 

*  Z  d  ,  r^£  describes  the  weighting  functions  in  (10)  applied  to  the  l  »  m  -  1 

previously  computed  basis  functions  4^  and  where  d^  is  found  as  above 
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is  the  energy  normalization  constant. 


From  (10),  we  note  that  in  our  G-S  procedure:  depends  only  on  fi,  the  weighted  and 

normalized  contribution  of  f^  is  subtracted  from  f2  and  thus  >2  depends  only  on  fj  and  more- 
so  it  contains  the  f2  data  not  present  in  f^,  similar  remarks  apply  to  }m  in  terms  of  fm_i 
and  fm.  Thus,  our  G-S  procedure  decorrelates  the  '.fn;  input  data  set.  Moreso,  in  many  cases 
M  >  N,  in  which  case  our  G-S  procedure  is  quite  attractive  since  it  forces  M  *  N  without  the 
need  for  the  use  of  an  ad  hoc  technique  to  selectively  remove  M  -  N  basis  functions.  We  also 
note  that  a  different  \  im :  basis  set  will  result. if  a  different  ordering  of  the  ■.  fn;  is  used 
and  that  for  each  new  ordering  of  the;fn)  a  new  h  will  result.  However,  for  a  fixed  fn. 
data  set,  all  h  will  correspond  to  the  same  hyperplane  in  a  rotated  hyperspace  version  of 
the  original  space.  This  property  follows  directly  from  the  fact  that  a  unitary  transforma¬ 
tion  does  not  change  the  relative  distances  between  points  on  a  hyperspace.  V’e  conclude  by 
noting  that  the  output  correlation  Rfh(0)  is  evaluated  at  •  =  0,  and  that  this  corresponds 
to  the  case  of  no  shift  between  the  input  and  the  reference  function  not  to  the  value  of  the 
correlation  at  its  maximum  peak  location.  We  also  remind  the  reader  that  a  shifted  (trans¬ 
lated)  version  of  any  input  function.!^  represents  a  new  input  function  and  will  result  in 
a  new  {f_}  input  data  set  and  a  new  h  filter.  In  Section  5,  we  use  these  last  two  observa¬ 
tions  in  the  design  of  a  maximum  common  information  (MCI)  filter  to  improve  the  correlation 
plane  SNRq  performance  of  our  SDF  pattern  recognition  processor. 


K-L  techniques  (i3-161  can  also  be  used  to  diagonalize  amatrix  such  as  R  in  (8).  We  con¬ 
sider  this  approach  in  more  detail  in  [8J.  For  now,  we  only  note  that  it  requires  an  ade¬ 
quate  statistical  ensemble  and  that  it  provides  the  optimal  basis  set  of  reduced  dimension¬ 
ality.  This  can  greatly  expedite  the  off-line  calculations  necessary  to  generate  the  SDF 
function  h. 


In  this  paper,. we  will  concentrate  on  the  use  of  the  G-S  technique  in  (10)  to  determine 
the  SDF  function  h  to  satisfy  (7)  for  the  intra-class  pattern  recognition  problem  of  present 
concern.  The  procedure,  notation  and  terminology  we  use  follow.  Diagonalization  of  R  in 
(8)  yields  a  set  of  equations  for  the  in  terms  of  the  fn 


=  £dnnV 


(11) 


We  denote  the  matrix  with  element  values  d^  in  (11)  as  the  G-S  coefficient  matrix.  Solving 
(11),  we  obtain  an  expression  for  the  original  ?n  inputs  in  terms  ot  the  basis  functions 
as  in  (5).  We  denote  this  matrix  with  elements  a-,,,  in  (5)  as  the  target  matrix,  FT  (or  F^.GS 
if  the  G-S  diagonalization  technique  is  used) .  The  equal  output  correlation  requirement  in 
(7)  can  then  be  written  in  terms  of  FT  as 

FTh  -  RoG,  (12) 

where  u  is  the  unit  vector.  Computing  the  inverse  target  matrix  FT_1  (by  Gaussian  elimina¬ 
tion  or  other  techniques)  ,  applying  F.^  to  JIT)  and  recalling  that^F^"^  FT  =  l_  (the  identity 
matrix) ,  we  find  the  expression  from  which  h  can  be  obtained  is  ‘ 

h  =  F_1RpG.  (13) 

This  gives  h  in  terms  of  Substituting  (11)  into  (13)  we  obtain  the  equation  from  which 

we  can  determine  n  from  the  elements  of  the  original  input  set  t  f n } .  The  above  notation  and 
terminology  and  specific  forms  for  the  various  matrices  will  be  provided  in  our  experimental 
data  (Section  6) . 


5.  ADDITIVE  (ZERO-MEAN)  NOISE  EFFECTS 


We  now  consider  the  effect  of  zero-mean  additive  noise  on  the  selection  of  the  SDF  and 
the  performance  of  our  synthetic  MSF  correlator.  We  desire  a  new  correlation  plane  SNRq  ex¬ 
pression  and  will  present  a  simplified  hyperspace  model  of  this  case.  We  then  discuss  a  new 
maximum  common  information  (MCI)  filter  concept  we  use  to  improve  the  performance  of  the 
processor.  The  assumption  of  zero-mean  noise  is  realistic  since  the  mean  of  the  data  is  re¬ 
moved  in  the  digital  processing  of  IR  data  and  because  such  filtering  directly  occurs  in 
optical  MSF  synthesis.  We  will  consider  the  case  of  colored  noise,  background  noise  with 
non-zero  mean  value  and  false  targets  in  (8). 

The  noisy  input  is  f'  »  f  ♦  n,  where  n  is  the  noise  vector  with  elements  nn  that  are  ran¬ 
dom  and  independent  with. a  variance  Jnn.  Since  the  mean  <n>  of  the  noise  is  zero,  its 
average  energy  en  aVg  =  <n  •  n>  «  Igzn  and  we  can  thus  bound  the  noise  by  the  hypersphere  at 


^  —  r\  •“) 


the  origin  of  radius  o  =  K<n  •  n>l/2,  where  the  constant  K  is  determined  by  the  confidence 
with  which  we  know  the  variance  of  the  noise.  Since<§,:>  =  ? ,  we  can  similarly  bound  f'  to 
lie  within  a  sphere  of  radius  r  =  'n;  of  h  We  thus  modify  our  correlation  threshold  cri¬ 
teria  in  (7)  to  be  •  fi  =  R  ±  r  to  include  the  effects  of  noise.  The  corresponding  hyper¬ 
space  diagram  of  Figure  2  helps  to  show  these  issues  under  simplifying  conditions  such  as 
no  attention  to  the  distribution  of  n  within  the  sphere  at  the  origin  and  with  no  attention 
to  devising  the  optimal  filter  fi  for  a  specific  noise  distribution. 
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FIGURE  2.  Simplified  hyperspace  model  showing  the  effects  of  additive  noise 


An  appropriate  SNRg  measure  for  the  output  correlation  plane  pattern  is  thus 

l/2_  „  c  1/2 


SNRg  =  K  = 


(eh  -r) 

<fi.fi>1'2 


0 .  5e. 


<n*n> 


(14) 


In  (14),  we  have  noted  that  p  £  e^2  -  r  must  be  satisfied  to  avoid  overlap  in  Figure  2. 

From  an  analysis  of  the  probability  of  correct  registration  Pq  and  using  Bayesian  logic,  we 
find  that  the  optimum  choice  for  r  and  p  is  r  »  0.5ejf'2  =  p.  This  maximizes  Pc  and  both  the 
probability  with  which  the  noise  is  bounded  within  p  at  the  origi-  ind  within  r  of  the 
hyperplane.  From  (14),Awe  see  that  maximizing  the  energy  eu  of  the  filter  and/or  reducing 
the  noise  variance  <n  •  n>  will  improve  the  performance  of  the  filter.  We  now  investigate 
the  use  of  a  new  MCI  technique  and  bandpass  filter  (BPF)  preprocessing  to  achieve  improved 
SDF  performance. 


We  can  show  that  an  upper-bound  s^UB  on  the  energy  of  the  filter  is  =  ef  (the  energy 
of  the  minimum  energy  input  function  fra  in  the  input  data  set)  and  that  a  lower-bound  £hLB  * 
1/ (N  <l/en>),  whei  e  <l/en>  is  the  average  reciprocal  energy  in  the  input  data  set.  Vie  have 
also  shown  that  if  any  pair  of  inputs  and  in  (fn)  correlate,  then  will  be  increased. 
We  use  this  observation  and  our  remarks  in  Section  4  on  the  G-S  technique  to  optimally  shift 
all  of  the  (fn)  inputs  to  maximize  their  cross-correlation,  thereby  increasing  c^,.  Since 
this  operation  aligns  and  maximizes  the  common  information  in  the  input  data  set,  we  refer 
to  this  technique  as  MCI  and  we  call  the  resultant  fi  an  MCI  filter.  Since  any  one  shifted 
version  of  one  target  aspect  need  be  included  in  our  input  set  to  enable  all  shifted  versions 
of  that  aspect  to  be  recognized  by  the  resultant  filter  (the  shift-invariant  feature  of  the 
Fourier  transform  MSF  correlator  insures  this) ,  we  can  simply  select  the  shifted  input 
that  maximizes  the  filter  energy  and  hence  the  output  SNR  in  (14). 

A  preprocessing  operation  that  we  applied  prior  to  selection  of  the  basis  functions  and 
the  MCI  filter  is  a  bandpass  filter  (BPF).  This  technique  was  described  earlier  [6,7]  and 
thus  is  not  repeated  here.  We  note  only  that  its  use  is  quite  appropriate  for  IR  pattern 


e> 


recognition  where  the  .teen  value  is  not  reliable  and  where  high  spatial  frequency  data  is 
noise.  This  operation  decreases  the  variance  of  the  noise  and  maximizes  the  SNRq  in  (14). 

6.  EXPERIMENTAL  VERIFICATION 

To  apply  the  SDF,  MCI  and  BPF  pattern  recognition  techniques,  we  chose  the  problem  of 
recognizing  an  M-60  tank  independent  of  its  aspect  from  input  IR  imagery.  This  terminal 
homing  missile  guidance  problem  thus  incorporates  both  intensity  and  geometrical  differences 
between  the  input  and  reference  data.  As  our  training  set,  we  used  seven  IR  images  of  an 
M-60  tank,  each  being  a  different  aspect  view  of  the  target.  A  representative  set  of  these 
images  and  their  digitized  versions  (128x128  pixels)  are  shown  in  Figure  3. 


f^  -  Right  Side  (RS) 


f3  -  Front/Side  (F/S) 


f .  -  Rear  (R) 
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FIGURE  3.  Selected  input  imagery  in  (f  }  (left)  and  their  digitized 
reconstructions  (right) . 


As  described  in  Section  5  and  references  [6,7],  optical  weighted  MSF  synthesis  was  used 
to  determine  the  optimal  BPF  to  use.  This  BPF  was  then  applied  to  the  input  data  set  to 
improve  the  correlation  plane  SNRg  in  (14) .  The  correlation  planes  for  all  49  image  pair 
correlations  were  then  obtained  and  the  locations  of  three  largest  correlation  peaks  in  each 
of  the  21  upper  off-diagonal  Rf.*  .  correlation  plane  patterns  were  noted.  A  sequential 
technique  was  used  to  investigate-3 the  filter  energy  corresponding  to  each  of  the  2  •  3 
(N  -  1) !  =  (66)!  =  4320  possible  input  shift  cases.'  The  combination  of  shifted  inputs  with 
the  maximum  filter  energy  £h  was  then  selected  as  our  input  data  set  ( f n ; .  The  autocorrela¬ 
tion  matrix  for  this  MCI  input  set  if-}  is  shown  in  Table  1.  The  G-S  coefficient  matrix 
obtained  from  Table  1  using  (10)  is  shown  in  Table  2. 

The  SDF  filter  h  that  results  from  applying  the  procedure  in  (11)  —  (13)  is  shown  in  Figure 
4.  In  Figure  5,  we  show  five  cross-sectional  scans  taken  at  one  pixel  intervals  through  the 
auto-correlation  output  for  input  image  lj  and  the  cross-correlation  output  •  h  of  this 
image  and  the  MCI  SDF  filter  h.  From  Figure  5,  we  see  that  the  shape  and  width  of  the  out¬ 
put  correlation  is  preserved  quite  well  by  our  MCI  SDF  reference  MSF- filter. 
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Table  1.  Auto-correlation  natrix 


Table  2.  Gram-Schmidt  coefficient  natrix 


INPUT  IMAGE  NUMBEK 

1  2  3  4  5  6  7) 

Ip  (relative  units) 

7.89  7.60  7.15  7.51  7.82  7.91  7.29  j 

Table  3.  Peax  intensities  for  the  correlations  of 
inputs  to  f?  with  n 
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FIGURE  4.  The  maximum  common  information  (MCI)  synthetic 
discriminant  function  (SDF)  filter  fi  obtained 
by  the  Gram-Schmidt  (G-S)  procedure. 


FIGURE  5.  Cross-sectional  scans  through  the  autocorrelation  of 
?3  •  (Figure  a)  and  the  cross-correlation  f3  •  h. 
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In  Table  3,  we  list  the  peak  correlation  values  ID  obtained  from  the  correlations  of  h  in 
Figure  4  with  all  seven  input  tank  aspects  £^-£7  (see  Figure  3) .  We  find  all  seven  I D 
values  to  be  equal  within  5%  as  predicted  from  (7) .  The  small  differences  observed  are  due 
to  sampling  and  roundoff  errors.  In  view  of  the  fact  that  the  one  SDF  can  recognize  all 
different  aspect  views  of  the  target,  the  variations  in  Table  3  are  negligible.  The  desired 
performance  of  our  MCI  SDF  filter  has  thus  been  demonstrated.  By  inspection  of  Figure  4,  we 
see  that  the  filter  function  h  exhibits  quite  significant  structure.  We  thus  expected  to 
exhibit  considerable  discrimination  [8]  for  inter-class  pattern  recognition  as  well  as  good 
intra-class  recognition  of  different  aspect  views  of  the  same  object  (the  case  considered 
herein) . 


SUMMARY  AND  CONCLUSIONS 

In  this  paper,  we  have  shown  how  to  formulate  the  conventional  matched  spatial  filter 
correlator  in  terms  of  a  new  hyperspace  representation.  We  have  also  included  the  case  of 
additive  zero-mean  noise  in  our  analysis.  This  representation  leads  to  a  new  synthetic  dis¬ 
criminant  function  matched  filter  capable  of  recognizing  a  reference  object  independent  of 
the  input  image  aspect  view  used.  The  use  of  bandpass  filter  preprocessing  to  enhance  the 
filter's  performance  on  IR  and  multisensor  imagery  was  noted  and  demonstrated.  A  new  maxi¬ 
mum  common  information  filter  concept  was  developed  to  enhance  the  correlation  SNRq  by  in¬ 
creasing  the  filter's  energy.  Experimental  verification  on  infrared  M-60  tank  imagery  was 
included  to  demonstrate  the  new  theoretical  concepts  described.  Kith  one  synthetic  dis¬ 
criminant  function  filter,  we  were  able  to  successively  correlate  and  recognize  all  aspect 
views  of  the  input  object  with  essentially  identical  output  peak  intensity  and  SNR  values. 
The  problem  and  application  were  selected  so  that  off-line  filter  synthesis  was  realistic. 
Methods  to  decrease  the  computational  load  and  storage  requirements  of  this  off-line  filter 
synthesis  process  are  the  subject  of  current  research. 
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ABSTRACT 

A  hvperspace  formulation  of  matched  spatial  filter  pattern  recognition  together  with  band¬ 
pass  filter  preprocessing  leads  to  synthesis  of  a  synthetic  discriminant  function  that  can 
recognize  a  reference  object  independent  of  intensity  and  geometrical  differences  between 
inputs.  Use  of  a  .  aximum  common  information  preprocessing  concept,  Karhunen-Loeve  tech¬ 
niques,  non-unitary  transformations  with  multi-channel  synthetic  discriminant  functions  and 
a  new  decorrelation  transf  irmation  to  provide  inter-class  discrimination  for  our  synthetic 
discriminant  function  syst  a  are  considered.  Experimental  verification  on  M-60  tank  targets 
and  an  armored  personnel  ca-  tier  false  target  are  included. 

1.  INTRODUCTION 


The  recognition  of  the  presence  of  a  reference  object  in  a  cluttered  background  and  the 
location  of  its  position  are  the  classic  problems  in  pattern  recognition  [1].  One  of  the 
more  attractive  solutions  to  such  problems  is  to  correlate  the  on-line  input  image  with  a 
stored  reference  pattern.  The  correlation  operation  can  be  realized  both  optically  (2]  and 
digitally  [3].  Our  present  concern  is  the  fact  that  in  most  practical  pattern  recognition 
problems,  the  input  image  and  the  reference  function  will  differ  due  to  intensity  variations 
(this  occurs  when  multisensor  or  infrared  imagery  is  used)  and  moreso  due  to  geometrical 
differences  that  arise  (especially  when  the  aspect  angle  from  which  the  object  is  viewed 
cannot  be  controlled) .  Our  specific  pattern  recognition  case  study  concerns  the  recognition 
and  location  of  tank  targets  from  IR  imagery  and  the  discrimination  of  these  target  objects 
from  false  targets  such  as  trucks  and  armored  personnel  carriers.  In  this  application,  both 
geometrical  and  intensity  differences  exist  between  the  input  and  reference  data. 

Because  of  the  ease  with  which  a  correlation  can  be  performed  optically,  we  consider  a 
new  hybrid  optical/digital  pattern  recognition  system.  This  processor  uses  bandpass  filter 
(BPF)  preprocessing,  an  optical  correlator  and  a  synthetic  discriminant  function  (SDF)  as 
the  matched  spatial  filter  (MSF)  reference.  The  SDF  fi  is  synthesized  off-line  from  a  train¬ 
ing  set  containing  N  different  aspect  views  { f n }  of  the  reference  object.  A  hyperspace  de¬ 
scription  of  the  data  base,  MSF  and  output  correlation  was  used  in  [4]  to  develop  an  SDF 
capable  of  recognizing  a  reference  object  independent  of  its  aspect  view.  Because  the  per¬ 
formance  of  a  correlator  rapidly  degrades  with  even  simple  geometrical  distortions  between 
the  input  and  the  reference  object  [5],  use  of  an  SDF  and  a  hypezspace  description  of  a 
pattern  recognition  problem  is  most  useful.  It  also  represents  a  new,  attractive  and  prac¬ 
tical  approach  to  optical  pattern  recognition  when  severe  geometrical  distortions  (such  as 
different  aspect  views  of  the  object)  must  be  handled. 

In  (6,7],  a  Gram-Schmidt  (G-S)  technique  for  SDF  synthesis  and  the  use  of  BPF  preproces¬ 
sing  for  the  case  of  IR  input  imagery  was  described.  In  [4],  we  described  our  hyperspace 
formulation  of  a  MSF  correlator  and  introduced  a  new  maximum  common  information  (MCI)  filter 
synthesis  concept.  In  this  paper,  we  consider  advanced  techniques  to  provide  improved  dis¬ 
crimination  performance  of  these  SDFs  fof  inter-class  pattern  recognition  applications.  We 
briefly  review  our  hyperspace  formulation  (Section  2)  and  our  SDF  synthesis  technique  to¬ 
gether  with  details  of  the  G-S  and  Karhunen-Loeve  (K-L)  methods  (Section  3) .  The  MCI  con¬ 
cept  together  with  its  use  in  aligning  common  image  structure  is  described  in  Section  4  with 
emphasis  on  the  improved  discrimination  performance  this  operation  provides.  Two  multi¬ 
channel  SDF  systems  for  inter-class  discrimination  are  highlighted  in  Section  5.  A  new 
general  decorrelation  transformation  and  its  use  in  inter-class  discriminationare  described 
in  Section  6.  These  latter  systems  utilize  non-unitary  transformations  to  provide  dimen¬ 
sionality  reduction  of  the  area  (volume)  of  the  discriminant  hypersurface .  Experimental 
verifications  are  included  in  Sections  4  and  6. 

We  distinguish  three  types  of  noise  for  which  we  desire  increased  discrimination.  These 
include:  additive  zero-mean  noise  (the  use  of  a  BPF  and  a  MCI  filter  is  quite  useful  for 
this  case  [4]);  color  background  noise  with  non-zero-mean  and  with  a  deterministic  nature; 
and  false  targets.  A  K-L  technique  (Section  6)  and  a  modified  G-S  technique  using  a  decor¬ 
relation  transformation  (Section  6)  are  found  to  provide  quite  excellent  discrimination  for 
these  latter  two  types  of  noise  and  false  targets. 


2.  HYPERSPACE  DESCRIPTION  OF  A  MSF  CORRELATOR 

We  consider  N  input  images  \fn',  each  corresponding  to  a  different  aspect  view  of  a  ref¬ 
erence  object.  We  describe  them  by 

M 

f  =  2  a„„i  ,  (X) 

n  m=i  n®  m 

where  the  %  are  basis  functions  in  the  set  {$mi  and  the  coefficients  amAare  the  elements 
of  the  target  matrix  Fy.  We  desire  to  determine  an  SDF  filter  function  h,  which  we  write 
as  a  similar  expansion  in  terms  of  the  same  basis  function  set  as 

M 

h  =  t  b  4  ,  (2) 

m=l  m  m 

such  that  the  correlation  Rf^of  an  fn  and  h  equals  a  constant.  We  describe  h  and  the  f  as 
M-dimensional  vectors  fi  and*'  fn  in  an  M-dimensional  hyperspace  whose  axes  are  the  basis 
functions  5m  and  the  coefficients  of  the  vectors  are  their  projections  on  the  axes,  e.g. 

f  =  (a1,a2,...,aM).  (3) 

We  can  write  our  correlation  condition,  f  ©  h  =  constant,  as 


h  =  I  a__  b  =  a  b 
m  run  m  «-  — • 


f©h 


Rfh(0)  =  Rp. 


The  condition  f  •  h  =  R  =  constant  in  (4)  describes  a  hyperplane  (if  h  is  fixedly.  Spe¬ 
cifically,  the  MSF  or  SDF  iris  a  vector  from  the  origin  in  hyperspace  normal  to  the  fn*  h  * 
Rp  hyoerplane  surface.  If  any  input  f  satisfies  (4),  then  we  say  that  it  i3  a  member  of  the 
set  (fn;  of  true  targets.  The  energy  of  the  function  f  is  defined  as 

e.  =  /f2(x)  dx  =  £  a2  =  f  •  f  =  R  .  (5) 

t  m  m  s 

If  «  f  ■  f  equals  a  constant  Rs  as  in  (5) ,  then  this  describes  a  hypersphere  or  an  equi- 
energy  surface  in  hyperspace.  Hyperplanes  and  hyperspheres  represent  two  quite  useful  and 
easily  described  and  realizable  dot  product  functions  of  use  in  pattern  recognition. 

3.  SYNTHETIC  DISCRIMINANT  FUNCTION  (SDF)  SYNTHESIS 

In  this  section,  we  highlight  how  the  filter  h  in  (2)  that  satisfies  (4)  for  the  input 
set  i f n }  in  (1)  can  be  derived  (4,6,7].  We  first  form  the  autocorrelation  matrix 

Rf  f  (0)  (6) 

i  j  J 

of  all  f  input  jmage  pairs.  We  then  diagonalize  R  using  G-S  [4J  or  K-L  [8,9]  techniques. 
This  yields  the  4>m  as 

J  »  Id  !  ,  (7) 

m  n  mn  n 

where  the  d^  are  the  elements  of  the  G-S  coefficient  matrix.  The  target  matrix  FT  and  its 
elements  a,,,,,  in  (1)  are  found  from  the  solution  of  (7.)  .  We  write  (3T  ai 

FTfi  =  RpU,  (8) 

where  u  is  the  unit  vector.  We  compute  the  inverse  target  matrix  F”1  and  apply  it  to  (8)  to 
obtain 

h  »  f"1!^,  i9) 


Substituting  (7)  into  (9),  we  find  h  in  terms  of  the  original  input  set  i f n  *  of  images. 
Our  G-S  algorithm  [4]  to  obtain  the  (»n:  from  the  { f n ?  is  described  by 


>2  =  I  f  2  (x)  -  Z2(r21)*1(x)  1/E. 


m-1 

=  If  (x)  -  :  Z_(r  j)  4>»  (x)  1/E 

m  m  £  =  i  m  mt  c  m 


(10) 


In  (10),  Zm  is  a  function  of  the  elements  of  the  autocorrelation  matrix  r^  (it  describes 
the  weighting  to  be  applied  to  the  previously  computed  basis  functions)  and  is  an  energy 
normalizing  constant. 

A  K-L  matrix  diagonalization  technique  (8,91  can  also  be  used  to  determine  the  •  j  .  In 
this  technique,  an  ensemble  average  matrix 


R  =  E  ( a  a  ; 
—  n  nm  nm 


(11) 


is  formed  where  the  a^  are  the  coefficients  of  the  fn<j,  elements  of  (f  }  in  any  orthonormal 
basis  set  ;m  and  where  the  ensemble  average  is  formed  over  the  image  index  n  (i.e.  over  all 
input  imagery) .  A  transformation  T  is  applied  to  R  to  form  the  diagonal  matrix 

*  1XRT.  (12) 


This  transformation  produces  new  inputs  fnKL  =  fna>T  and  new  basis  functions  =  imT  that 

satisfy 


-$mKL  =  'm^mKL' 


(13) 


i.e.  JmKL  are  the  eigenvectors  of  R  and  the  Aln  are  the  eigenvalues. 

Use  of  a  Fourier  kernel  basis  function  set  is  also  possible  as  the  Fourier  transform  can 
easily  be  obtained  optically  and  digitally.  However,  use  of  this  basis  set  is  quite  restric¬ 
tive.  Its  main  function  occurs  in  BPF  processing.  The  G-S  and  K-L  techniques  are  far  more 
orgainized  and  less  ad  hoc  and  less  restrictive.  They  directly  provide  the  best  basis  func¬ 
tion  set  for  the  input  training  data  given.  In  the  G-S  technique,  the  basis  4^  is  a  function 
only  of  f^,  the  basis  $2  is  a  function  only  of  f2  (with  the  part  of  fj  contained  in  fj^  removed)  , 
etc.  This  operation  of ^decorrelation  of  the  input  data  performed  by  the  G-S  expansion  will 
prove  quite  useful  in  our  decorrelation  transformations  (Section  6)  for  improved  SDF  descrmi- 
nation  performance.  The  K-L  technique  is  superior  in  terms  of  dimensionality  reduction. 

Each  basis^f unction  in  the  K-L  expansion  is  a  function  of  all  inputs  in  (fn).  Thus,  we  can 
order  the  },j,kl  terms  °f  decreasing  eigenvalues  Xm.  By  retaining  only  those  basis  func¬ 
tions  with  the  largest  eigenvalues,  a  quite  reduced  dimensionality  problem  results.  In  prac¬ 
tice,  three  eigenvectors  are  adequate  to  describe  and  represent  over  95*  of  the  data  in  (fn:. 
In  Section  6,  we  combine  these  features  of  K-L  and  G-S  expansions  into  a  new  image  discrimi¬ 
nation  technique. 


4.  DISCRIMINATION  PERFORMANCE  OF  THE  MCI  SDF  FILTER 


In  (4],  we  used  an  image  data  base  consisting  of  seven  inputs  of  an  M-60  tank  taken  from 
different  aspects.  We  applied  a  BPF  to  the  data  (with  the  bandpass  determined  by  optical 
weighted  MSF  synthesis  as  in  [6,7]).  We  then  operated  on  this  BPF  data  base  to  increase  its 
common  information. 


To  see  the  reason  for  this, 
described  by 


we  recall  from  [4]  that 
1/2 


0 . 5ew 


SNR 


o  =  "“--171 

<n-n> 


the  SNR0  of  the  output  correlation  is 


(14) 


where  -:h  is  the  energy  of  the  filter  h  and  where  the  denominator  is  the  variance  of  the  input 
noise  (assumed  to  be  zero-mean  additive  Gaussian  noise) .  From  (14) ,  we  note  that  increasing 
Eh  increases  the  SNRg.  In  [41,  we  also  noted  that  if  the  correlation  Rf . £ .  between  any  two 
elements  of  (fni  is  increased,  30  is  the  filter's  energy.  From  Section  1  1  2,  we  note  that 
a  shifted  or  translated  version  of  any  input  corresponds  to  a  new  input,  a  new  hyperspace 
vector  and  that  a  new  h  results  for  each  such  shifted  input.  From  Section  3,  we  note  that 


if  the  input  set  - fn  ■  is  fixed,  then  the  same  h  will  result  regardless  of  the  basis  set  used. 
Mcreso,  any  one  translated  input  can  be  used  to  recognize  all  translated  versions  of  a  given 
target  aspect  (because  of  the  shift-invariance  of  a  Fourier  transform  correlator) . 

From  these  considerations,  we  proceeded  to  optimally  shift  all  input  image  pairs  to  maxi¬ 
mize  their  common  information  and  hence  their  cross-correlation.  The  resultant  MCI  input 
image  set  was  obtained.  Its  ayto-correlation  matrix  was  produced.  Its  G-S  coefficient 
matrix  computed  from  (10)  and  h  calculated  from  (7)  and  (9) .  This  ft  MCI  SDF  filter  (Figure 
1)  was  found  to  yield  nearly  equal  output  correlations  (within  5%)  with  ajl  aspects  of  the 
reference  M-60  tank  object.  By  inspection  of  Figure  1,  we  see  that  this  h  has  considerable 
structure  and  that  it  is  not  a  blurred  average  of  all  of  the  inputs.  Thus,  we  expected  to 
provide  good  discrimination  for  inter-class  pattern  recognition  (distinguishing  true  M-60 
targets  ft  from  false  targets  ff) . 


FIGURE  1.  Synthetic  discriminant  function  (SDF)  produced  by 
the  maximum  common  information  (MCI)  technique. 


FIGURE  2.  Synthetic  discriminant  function  (SDF)  produced 

by  colocating  the  centroids  of  the  input  image  set. 


To  pursue  this  observation,  we  formed  a  second  h  filter  from  the  same  BPF  image  set  by 
colocating  the  centroids  of  the  imagery  (Figure  2) .  This  represents  a  good  initial  estimate 
of  the  proper  shifts  necessary  for  the  input  data.  From  an  analysis  of  the  autocorrelation 
matrices  for  the  MCI  and  the  centroid  colocated  imagery,  we  found  that  the  sum  of  the  off- 
diagonal  elements  was  increased  by  over  20%  for  the  MCI  data  set.  The  peak  intensity  of  the 
correlation  outputs  obtained  3howed  about  a  15%  increase  in  the  filter  energy.  We  do  not  ex¬ 
pect  these  two  measures  to  be  the  same,  but  the  increased  filter  energy  and  hence  the  in¬ 
creased  SNRq  obtained  with  the  MCI  filter  is  clearly  apparent  from  this  experiment. 


We  expect  the  major  benefit  from  the  MCI  filter  to  come  from  its  enhanced  discrimination. 
To  investigate  this,  we  compare  Figures  1  and  2.  Although  the  centroid  filter  in  Figure  2 
appears  to  have  more  structure  than  the  MCI  filter  in  Figure  1,  this  is  quite  misleading, 
since  a  much  higher  exposure  was  necessary  to  photograph  Figure  2  than  Figure  1  and  since 
much  of  the  added  "structure"  present  in  Figure  2  is  present  because  the  image  pairs  were 
not  properly  aligned.  Thus,  in  Figure  2,  the  added  structure  is  of  low  intensity,  whereas 
all  intensities  in  Figure  1  are  much  higher  than  those  in  Figure  2.  In  addition,  much  more 
lower  intensity  structure  is  actually  present  in  Figure  1  (but  it  is  not  visible  because  of 
the  low  dynamic  range  of  the  available  display  used  to  produce  these  figures) . 
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Table  1.  Correlation  coefficient  ma¬ 
trix-foe-  the-  maximum  -common 
information  (MCI)  data  set. 


Table  2.  Correlation  coefficient 
matrix  for  the  centroid 
data  set. 


To  further  pursue  these  image  structure  and  high  intensity  features  of  the  MCI  filter,  we 
form  the  correlation  coefficient  matrices  of  both  image  sets  (Tables  1  and  2) .  We  denote  the 
elements  of  these  matrices  by  r[^  and  note  that  they  are  the  elements  ri  j  of  the  corresponding 
autocorrelation  matrix  normalized  by  the  square  root  of  the  product  of  the  energies  of  each 
input  image  pair.  These  r! .  are  thus  measures  of  the  common  structure  present  between  dif¬ 
ferent  input  image  pairs,  •'if  the  structures  in  images  i  and  j  are  identical,  then  r? j  *  1 
(autocorrelation) .  As  their  structures  differ,  r^j  decreases.  In  comparing  Tables  1  Jand 
2,  we  note  that  different  image  pairs  exhibit  a  larger  increase  due  to  MCI  than  do  others. 

The  F/S-R/S  and  the  F-R  image  pair  cross-correlations  are  boxed  for  comparison  in  Tables  1 
and  2.  These  image  pairs  show  large  (2.9  to  5.0)  improvement  gain  factors.  This  is  ex¬ 
pected  because  these  image  pairs  have  significant  common  structure  (as  an  inspection  of  the 
images  will  confirm) . 

Our  major  concern  is  that  this  structural  alignment  occurs  automatically  without  the  need 
to  use  merely  ad  hoc  techniques.  Moreso,  the  structural  alignment  in  an  MCI  filter  is 
weighted  by  the  energies  of  each  of  the  aligned  image  parts.  To  quantify  the  discrimination 
ability  of  the  MCI  SDF  filter,  we  use  the  reduced  set  of  two  tank  images,  f^  (RS,  right  side) 
and  fj  (R/S,  rear /side)  as  shown  in  Figures  3a  and  3b.  The  autocorrelation  matrix,  G-S  co¬ 
efficient  matrix,  target  matrix  and  inverse  target  matrix  for  these  data  were  then  formed 
(Figure  4).  From  this,  the  SDF 


h'  =  fx  +  0.46?2  (15) 

was  determined  using  R_  =  1.  The  correlation  of  this  filter  with  the  inputs  f,  and  fc  gave 
equal  output  correlation  values  of  RD  =  30.3.  We  then  correlated  this  same  SDF  filter  with 
a  typical  false  target,  an  armored  personnel  carrier  (ARC)  shown  in  Figure  3c.  For  this 
cross-correlation,  we  found  a  much  lower  correlation  peak  intensity  value  of  5.4  (or  7dB 
below  the  true  target  correlations) .  We  thus  conclude  that  the  discrimination  performance 
of  the  MCI  SDF  filter  is  quite  good  as  predicted. 
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3b 
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FIGURE  3.  Photographs  of  two  tank  targets,  (a)  fi,  RS 


(right  side) ,  (b)  f 
false  target  (c)  ff 
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,  R/S  (rear/side)  and  a 
(armored  personnel  carrier) . 


4c  4d 


FIGURE  4.  Data  matrices  for  the  imagery  in  Figures  3a  and 
3b.  (a)  Auto-correlation  matrix,  (b)  G-S  coeffi¬ 
cient  matrix,  (c)  Target  matrix  FT,  and  (d)  In¬ 
verse  target  matrix  F.T”  . 


5.  DISCRIMINATION  USING  MULTI-CHANNEL  SDFs  AND  NOW-UNITARY  TRANSFORMATIONS 

Conventional  unitary  transformations  do  not  yield  a  different  effective  h  filter  func¬ 
tion  and  hence  no  reduction  in  the  volume  of  the  discriminant  surface  will  occur.  Moreso, 
only  quite  restricted  discriminant  surfaces  (i.e.  a  hyperplane  or  a  hypersphere)  are  possible 
with  one  unitary  transformation.  In  this  section,  we  highlight  our  recent  work  on  the  use 
of  non-unitary  transformations  for  multi-channel  SDF  processors  to  provide  hyperspace  dis¬ 
criminant  surfaces  of  more  complex  shape  and  with  fewer  points  on  them.  This  should  yield 
better  discrimination  performance  for  the  SDF  system.  This  technique  is  of  considerable  use 
in  the  case  of  low  energy  false  targets  that  lie  within  the  neighborhood  of  the  original 
discriminant  hyperplane  described  by  (4) .  Elimination  of  such  noise  or  false  targets  ff  re¬ 
quires  a  hyperspace  discriminant  surface  of  reduce  area  (or  volume) .  Only  non-unitary  trans¬ 
formations  can  achieve  this  (since  the  filter  function  fi  is  effectively  the  same,  but  in  a 
rotated  hyperspace,  for  the  case  of  unitary  transformations)  and  a  multi-channel  SDF  system 
is  necessary. 

Specifically,  we  envision  an  input  target  set  f'ft)  with  a  SDF  filter  h.  Applying  a  non- 
unitary  transformation  T  to  (f»)  we  obtain  a  new  data  set  { f T }  and  a  new  fiT  filter.  If  we 
require  (ft)  to  satisfy-?!  and  (fiy)  to  satisfy  fi^,  then  only  a  reduced  class  of  inputs  will 
solve  both  filter  functions.  This  occurs  because  the  non-unitary  transformations  yield  new 
inputs  not  present  in  the  original  3et  of  data.  The  resultant  intersection  of  the  discrimi¬ 
nant  hypersurfaces  for  both  filters  (one  will  be  a  hyperplane  and  the  other  of  more  complex 
shape)  will  be  a  more  complex  surface  with  fewer  vector  points  included.  Thus,  fewer  input 
images  will  satisfy  this  processor  and  improved  discrimination  will  result.  We  have  con¬ 
sidered  two  such  non-unitary  transformations,  the  normalization  of  targets  f^ix,!)  *  f(x)/ef 
and  the  square-law  transformation  fT(x)  »  f2(x). 
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FIGURE  5.  Multi-channel  SDF  processor 
using  a  non-unitary  normali¬ 
zation  transformation.  The 
common  discriminant  hyper¬ 
surface  is  the  points  f^ 
and  f£. 


FIGURE  6.  Multi-channel  SDF  processor  using 
a  non-unitary  square-law  transformation. 

With  R_  =  R_iO . lRp,  the  common  discriminant 
hypersurfacl  is  the  double  cross-hatched  re¬ 
gion  shown. 


In  Figures  5  and  6,  we  show  the  intersection  of  these  two  multi-channel  discriminant 
surfaces.  The  second  surface,  produced  by  the  non-unitary  transformation  is  not  a  plane. 

In  the  case  of  the  normalized  transformation  (Figure  5) ,  it  is  the  unit  eaui-energy  (norma¬ 
lized  intensity)  sphere.  In  this  case  (with  no  noise) ,  the  discriminant  surface  is  simply 
two  points  corresponding  to  the  two  input  target  data  set  assumed.  In  the  second  case 
(Figure  6)  ,  the  square-lav;  transformation  corresponds  to  a  hyperspace  that  is  a  parabola  in 
the  original  basis  space.  The  resultant  discriminant  surface  (using  an  Rp+0.1Rp  threshold 
on  the  value  of  the  output  correlation)  is  shown  by  the  double  cross-hatcned  area  in  Figure 
6.  In  both  cases,  the  intersection  volume  of  the  discriminant  surface  is  reducedand  false 
targets  of  low  energy  and  those  false  targets  lying  close  to  the  original  discriminant 
hyperplane  will  not  be  recognized  in  this  new  multi-channel  system. 

6.  NON-UNITARY  DECORRELATION  TRANSFORMATION  FOR  IMPROVED  DISCRIMINATION 

As  noted  in  Sections  4  and  5,  only  non-unitary  transformations  can  change  the  effective 
hyperspace  representation  of  a  given  data  set.  From  our  correlation  plane  SNR  analysis  and 
our  hyperspace  description  (Section  2)  of  the  SDF  correlator,  we  see  that  a  reduced  hyper¬ 
space  area  or  volume  leads  to  a  pattern  recognition  system  with  better  discrimination.  Thus, 
in  this  section,  we  consider  use  of  a  new  decorrelation  transformation  that  can  be  applied 
to  a  G-S  or  a  K-L  image  basis  set  to  remove  selected  basis  functions  and  hence  selected  in¬ 
puts.  This  operation  thereby  decreases  the  dimensionality  of  the  hyperspace  and  thus  im¬ 
proves  the  discrimination  performance  of  the  processor  by  removing  false  target  and  noise 
data.  For  our  experiments,  we  chose  a  G-S  basis  set  since  the  basis  function  $n  in  this 
representation  is  a  function  of  the  input  fm  alone.  In  a  K-L  basis  set,  each  4>ki,  is  a  func¬ 
tion  of  all  of  the  input  images.  Thus,  a  G-S  representation  is  more  conducive  for  direct 
use  of  our  proposed  non-unitary  transformation,  since  it  directly  removes  selected  basis 
functions  and  hence  selected  inputs  (e.g.  false  targets  ff) .  A  K-L  basis  set  representation 
is  of  utmost  use  in  reducing  the  dimensionality  of  the  data.  Techniques  to  obtain  separate 
K-L  basis  functions  corresponding  to  two  different  classes  of  inputs  have  been  advanced  [9) 
and  are  appropriate  in  advanced  versions  of  our  decorrelation  transformation.  Lack  of  a  suf¬ 
ficiently  large  statistical  ensemble  of  true  ft  and  false  ff  target  data  precluded  use  of 
this  K-L  transformation  in  our  present  experiments. 

Our  proposed  non-unitary  decorrelation  transformation  concept  is  ouite  simple.  We  de¬ 
scribe  it  for  a  simple  case  and  then  extend  the  concept  to  the  more  general  problem.  We 
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assume  that  the  first  basis  function  corresponds  to  a  false  target  ff  (in  the  G-S  tech¬ 
nique,  this  is  easily  achieved  by  picking  fi  =  ff) and  that  the  remaining  N  -  1  basis  func¬ 
tions  correspond  to  true  targets  ft  (i.e.,  in  the  G-S  technique,  we  select  f2-f>j  to  be  mem¬ 


bers  of 


We  apply  the  non-unitary  decorrelation  transformation  u  to  the’  target  matrix 


Ft  to  obtain  a  new  target  matrix  Fy  as  below 


^  = 


°i.O 

o'1 


0  aN2*  *  *aNM 


From  (16) ,  we  see  that  the  decorrelation  transformation  u  has  yielded  a  new  target  matrix 
with  all  information  associated  with  the  first  basis  function  (and  hence  the  false  target 

input  f^  =  ff)  removed.  The  new  h1  calculated  from  this  Fy  will  thus  not  contair  any  of  the 
false  target  energy  associated  with  23  =  ?f-  Its  performance  and  discrimination  will  thus 
be  improved.  We  can  directly  extend  this  technique  to  larger  {ff}  data  sets  by  letting  the 
first  J  input  functions  f^ — » f j  in  {fn}  in  the  G-S  procedure  correspond  to  {ff-  and  the  last 
N  -  J  elements  correspond  to  (ffc).  K-L  techniques  [9]  can  also  be  developed  to  produce  sepa¬ 
rate  basis  functions  corresponding  to  different  image  classes.  (In  this  case,  a  greatly  re¬ 
duced  dimensionality  is  sufficient  to  represent  the  true  and  false  target  data  sets) . 
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FIGURE  7.  Data  matrices  for  the  decorrelation  transformation  on  the  data  of  Figure 
3.  (a)  Auto-correlation  matrix,  (b)  G-S  coefficient  matrix,  (c)  Full 

target  matrix  Fy,  (d)  reduced  target  matrix  F\£,  and  (e>  inverse  of  re¬ 
duced  target  matrix  F,J,-1. 

To  demonstrate  the  use  of  this  technique,  we  return  to  the  examples  in  Figures  3  and  4. 

We  use  the  APC  (Figure  3c)  as  ff  (our  first  function)  and  the  two  tank  images  (Figures  3a  and 
3b)  as  our  second  and  third  input  functions.  In  Figure  7,  we  show  the  autocorrelation  matrix, 
G-S  coefficient  matrix,  full  target  matrix  Fy,  reduced  target  matrix  Fy,  and  the  inverse  F’”l 
of  the  reduced  target  matrix  for  these  inputs.  The  reduced  target  matrix  Ff  is  obtained  By 
applying  r-  -i 


0  0  0 
u  «  0  10 

0  0  1 


It  thus  consists  of  the  lower  two  rows  and  columns  of  Fy.  The  new  filter  computed 


from  this  Fy  is 


fx  +  0.42  fg  -  0.10  ff. 


This  new  filter  function  was  calculated,  synthesized  and  correlated  with  the  f^,  fj  and  ff 
inputs.  When  applied  to  the  true  target  inputs  (f]_  and  2c),  an  R  =  30.0  value  was  obtained. 
This  represents  only  a  negligible  1%  loss  from  the  RD  =  30.3  value  obtained  in  Section  4. 

For  the  cross-correlation  of  h’  with  the  false  target  2f,  Rp  »  0  wag  obtained  at  T  =  0.  We 
can  widen  the  null  in  the  2f  •  fi'  correlation  output  by  using  more  ff  inputs  (each  being  a 
shifted  version  of  the  ff  false  target  used) .  For  our  case,  five  such  inputs  would  suffice 
to  null  all  shifted  versions  of  if. 

1.  SUMMARY  AND  CONCLUSIONS 

In  this  paper,  we  have  feviewed  our  hyperspace  description  of  a  matched  spatial  filter 
(MSF)  correlator,  the  concept  of  the  use  of  a  synthetic  discriminant  function  (SDF)  as  the 
MSF  reference  and  several  techniques  for  generating  the  basis  function  sets  necessary  to 
obtain  the  SDF.  As  before,  bandpass  filter  preprocessing  is  used  to  remove  the  effects 


of  intensity  differences  in  the  data  (a  problem  that  arises  with  I®  and  multi sensor  imagery). 
A  maximum  common  information  (MCI)  filter  concept  used  to  maintain  intra-cl; ss  recognition 
of  different  aspect  views  of  the  reference  object  with  one  MCI  SDF  filter  w,  s  analyzed  for 
its  ability  to  also  perform  inter-class  discrimination  between  true  and  false  targets.  The 
structure  and  intensity  of  the  MCI  SDF  was  found  to  be  excellent  and  its  ability  to  suppress 
false  target  correlations  was  measured  and  seen  to  be  good. 

Non-unitary  transformations  to  produce  hyper surfaces  with  a  reduced  number  of  data  points 
and  with  more  complex  shapes  were  then  described.  A  multi-channel  SDF  system  was  shown  to 
be  quite  useful.  Two  examples  of  such  systems  were  discussed.  A  new  decorrelation  trans¬ 
formation  was  then  presented  that  allowed  specific  false  targets  (or  colored  background  noise 
of  non-zero  mean)  to  be  eliminated  from  the  data  set.  Experimental  verification  on  IR  imagery 
of  different  aspect  views  of  an  M-60  tank  and  an  armored  personnel  carrier  false  target  were 
obtained  with  excellent  results. 

The  advanced  discrimination  techniques  described  and  demonstrated  in  this  paper  make  the 
use  of  SDKs  in  optical  or  digital  MSF  correlators  even  more  attractive  and  practical  than 
they  initially  were. 
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March  1981,  39-47  (Casasent), 

22.  "Pattern  Recognition:  A  Review,"  IEEE  Spectrum,  28-33,  March  1981 
(Casasent). 

23.  "Processing  Flexibility  by  Hybrid  Optical/Digital  Techniques,"  Proc.  Work¬ 
shop  of  Future  Directions  in  Optical  Data  Processing,  Texas  Tech.  Rept., 

1  March  1981,  1723  (Casasent  and  Kumar). 
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9.  PUBLICATIONS 


9.1  PUBLICATIONS  (FY  1982) 

Publications  from  1  September  1980  -  1  September  1981  on  work  performed 
under  AF0SR-79-0091  follow. 


9.1.1  PAPERS  PUBLISHED 


1.  "Hybrid  Processor  to  Compute  Invariant  Moments  for  Pattern  Recognition," 
Opt  Lett,  _5,  395-7,  September  1980  (Casasent  and  Psaltis) . 

2.  "Optical  Work  Recognition,  Case  Study  in  Coherent  Optical  Pattern  Recog¬ 
nition,"  Opt  Engr,  19,  716-721,  September  1980  (Casasent  et  al). 

3.  "Lensless  Matched  Spatial  Filter  Correlator  Experiments,"  Opt  Commun,  34 , 
311-315,  September  1980  (M.  Shen  et  al). 

4.  "HOE/Lensless  Matched  Spatial  Filter  Wavelength-Scaling  Correlator,"  Opt 
Commun,  34,  316-320,  September  1980  (M.  Shen  et  al). 

5.  "A  Laser  Diode/Lensless  MSF  Optical  Pattern  Recognition  System,"  EOSD, 
46-52,  November  1980  (Casasent  et  al). 

6.  "Optical  Pattern  Recognition:  Matched  Spatial  Filter  Processors,"  EOSD, 
November  1980,  33-39  (Casasent). 

7.  "Optical  Pattern  Recognition:  Beyond  Matched  Spatial  Filtering,"  EOSD, 
March  1981,  39-47  (Casasent). 

8.  "Pattern  Recognition:  A  Review,”  IEEE  Spectrum,  28-33,  March  1981 
(Casasent) . 

9.  "Processing  Flexibility  by  Hybrid  Optical/Digital  Techniques,"  Proc.  Work¬ 
shop  of  Future  Directions  in  Optical  Data  Processing,  Texas  Tech.  Rept., 

1  March  1981,  17-23  (Casasent  and  Kumar). 

10.  "Binarization  Effects  in  a  Correlator  with  Noisy  Input  Data,"  Appl  Opt, 

20,  1433-1438,  April  1981  (Kumar  and  Casasent). 

11.  "Beyond  Matched  Filtering,"  Israel  J.  Techn  (Accepted)  (Casasent). 

12.  "A  Laser  Diode/HOE  Pattern  Recognition  System,"  Academia  Sinica,  (Accepted) 
(Casasent) . 

13.  Y.  Barniv,  H.  Mostafavi  and  D,  Casasent,  "Correlation  of  Images  with 
Random  Contrast  Reversals,"  SPIE,  238,  156  (July  1980). 
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14.  D.  Casasent,  S.  Eiva  and  B.  Kumar,  "linage  Quality  Effects  in  Optical 
Correlators,"  SPIE,  310  (August  1981). 

15.  Y.  Barniv  and  D.  Casasent,  "Multisensor  Image  Registration:  Experimental 
Verification,"  SPIE,  292  (August  1981). 

16.  C.  Hester  and  D.  Casasent,  "Intra-Class  IR  Tank  Pattern  Recognition  Using 
SDFs , "  SPIE,  292,  (August  1981). 

17.  C.  Hester  and  D.  Casasent,  "Inter-Class  Discrimination  Using  SDFs,"  SPIE, 
302  (August  1981). 


9.1.2  PAPERS  SUBMITTED 


18.  "Input  Quantization  Effects  in  an  Image  Correlator,"  Appl  Opt,  Submitted 
(Kumar  and  Casasent). 

19.  D.  Casasent,  F.  Caimi  and  A.  Khomenko,  "The  Soviet  Priz  Spatial  Light 
Modulator,"  Applied  Optics,  Rapid  Communications  (accepted  1981). 

20.  D.  Casasent,  F.  Caimi  and  A.  Khomenko,  "Test  and  Evaluation  of  the  Soviet 
Prom  and  Priz  Spatial  Light  Modulators,"  Applied  Optics  (submitted  1981). 

21.  D.  Casasent  and  C.  Neuman,  "An  Iterative  Optical  Processor:  Selective 
Survey  of  Operations  Achievable,"  Proceedings  NASA  Langley  Conference  on 
Optical  Information  Processing  (August  1981). 

22.  M.  Carlotto  and  D.  Casasent,  "An  Iterative  Optical  Vector-Matrix  Processor," 
SPIE  Institute  Series,  (submitted  1981). 

23.  M.  Carlotto  and  D.  Casasent,  "A  Microprocessor-Based  Fiber-Optic  Iterative 
Optical  Processor,"  Applied  Optics  (submitted  1981). 


9.2  SEMINARS  AND  PRESENTATIONS  (1  October  1980  -  31  September  1981) 

1.  "Optical  Data  Processing",  Carnegie-Mellon  University,  Sophomore  Seminar, 
(October  1980) . 

2.  "Airborne  Optical  Pattern  Recognition  System  Design",  EOSD  Conference, 
Boston  (November  1980). 

3.  "Optical  Pattern  Recognition",  General  Dynamics,  Pomona,  California 
(December  1980) . 

4.  "Optical  Pattern  Recognition",  Northrop  Corporation,  Anaheim,  California 
(December  1980). 
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5.  "Optical  Pattern  Recognition",  Hughes  Research  Corporation,  Malibu,  Cali¬ 
fornia  (December  1980) . 

6.  "Optical  Pattern  Recognition",  BDM  Corporation,  Virginia  (February  1981). 

7.  "Optical  Vector-Matrix  Processors,"  Washington,  D.C.,  NRL  2-D  Signal 
Processing  Conference,  (March  1981). 

8.  "Optical  Pattern  Recognition",  Eglin  Air  Force  Base,  Florida  (March  1981). 

9.  "Optical  Pattern  Recognition",  Ball  Aerospace  Corporation,  Boulder,  Colorado, 
(March  1981). 

10.  "Optical  Data  Processing",  CMU,  Pittsburgh,  Management  Program  (April  1981). 

11.  "Optica]  Data  Processing  at  CMU”,  OSA  Meeting,  Pittsburgh  (April  1981). 

12.  "Optical  Processors  for  Robotics  Applications",  Westinghouse  Corporation, 
Pittsburgh  (May  1981). 

13.  "Optical  Processors  for  Robotics  Applications",  CMU,  Pittsburgh  (May  1981). 

14.  "Use  of  Synthetic  Discriminant  Functions  in  Optical  Pattern  Recognition", 
General  Dynamics,  Pomona,  California  (August  1981). 

15.  "Design  of  and  Operations  Achievable  on  an  Iterative  Optical  Processor", 
Anaheim,  California  (August  1981). 

16.  "Operations  Achievable  on  an  Iterative  Optical  Vector-Matrix  Processor", 

NASA  Conference  on  Optical  Information  Processing  for  Aerospace  Applications, 
NASA  Langley,  Virginia  (August  1981). 

17.  "Optical  Pattern  Recognition  for  Missile  Guidance",  General  Dynamics,  Convair 
Division,  San  Diego,  California  (August  1981). 

18.  "Optical  Pattern  Recognition  for  Industrial  Applications",  TRW  Corporation, 
Los  Angeles,  California  (August  1981). 

19.  "Intra-Class  Pattern  Recognition  Using  Synthetic  Discriminant  Functions", 

SPIE  Conference,  San  Diego,  California  (August  1981). 

20.  "Inter-Class  Discrimination  Using  Synthetic  Discriminant  Functions",  SPIE 
Conference,  San  Diego,  California  (August  1981). 

21.  "Multi-Sensor  Optical  Pattern  Recognition:  Experimental  Verification",  SPIE 
Conference,  SPIE,  San  Diego,  California  (August  1981). 

22.  "Image  Quality  Effects  on  Optical  Pattern  Recognition",  SPIE  Conference, 

San  Diego,  California  (August  1981). 


23.  "Optical  Information  Processing  at  CMU  in  EE",  CMU,  Pittsburgh,  Sophomore 
Seminar  (September  1981). 
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9.3  THESES  SUPPORTED  BY  AFOSR  (October  1980  -  September  1981) 

1.  Hiroyasu  Murakami,  M.S.  Dissertation,  "Matched  Filter  Statistical  Correlator" 
(February  1981) . 

2.  Saulius  Eiva,  M.S.  Dissertation,  "Image  Quality  Effects  in  Optical  Cor¬ 
relators"  (May  1981). 

3.  C.  Hester  Ph.D.  Dissertation,  "Synthetic  Filters  for  Multiclass  Pattern 
Recognition"  (May  1981) . 

4.  Y.  Barniv,  Ph.D.  Dissertation,  "Multi-Sensor  Image  Registration"  (May  1981). 

5.  M.  Carlotto,  Ph.D.  Dissertation,  "Iterative  Electro-Optic  Matrix  Processor" 
(May  1981). 

6.  A.  Sexton,  M.S.  Dissertation,  "Digital  Analysis  of  Space-Variant  Optical 
Processors"  (July  1981). 

9.4  PATENTS  (1  October  1980  -  31  September  1981) 

Four  patent  disclosures  were  produced  during  the  past  year.  The  titles 
of  these  follow. 

(1)  Multiple-Invariant  Space-Variant  Pattern  Recognition  System, 

(2)  Pattern  Recognition  by  Invariant  Moments, 

(3)  Synthetic-Discriminant  Functions  for  Multi-Class  Pattern  Recognition, 

(4)  Equalization  and  Coherence  Measure  Correlator. 

A  patent  has  been  submitted  on  item  (3)  above. 

AF  invention  number  14,509  "Multi-Variant  Technique  for  Multi-Class  Pattern 
Recognition." 

9.3  PUBLICATIONS  TO  DATE  (1  October  1979  -  30  September  1981) 

Publications  on  research  sponsored  by  AFOSR-79-0091  grant  (1  October  1979  - 
30  September  1980)  follow.  The  FY81  list  of  publications  is  contained  in  Sec¬ 


tion  9.1. 
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1.  "Photo-DKDP  Light  Valve  in  Optical  Data  Processing,"  Appl  Opt,  18 , 
3307-3314,  October  1979  (Casasent  and  Luu). 

2.  "Coherent  Optical  Pattern  Recognition,"  Nikkei  Electronics,  150-181, 
October  1979,  In  Japanese  (Casasent). 

3.  "Optical  Data  Processing  for  Advanced  Missile  Guidance  Needs,"  AIAA, 
October  1979  (Casasent). 

4.  "Spread  Spectrum  Optical  Signal  Processors,"  Proc  EOSD,  333-342, 

October  1979,  (Casasent  and  Psaltis). 

5.  "Space  Blue  Bandwidth  Product  in  Correlator  Performance  Evaluation," 

J  Opt  Soc  Amer,  70,  103-110,  January  1980,  (Kumar  and  Casasent). 

6.  "Optical  Image  Processing,"  EOSD,  Tokyo,  January  1980,  In  Japanese 
(Casasent) . 

7.  "Optical  Signal  Processing,"  EOSD,  Tokyo,  January  1980,  In  Japanese 
(Casasent) . 

8.  "Beyond  Matched  Filtering,"  Opt  Engr,  19 ,  152-156, (March  1980)  February 
1980  (Caulfield  et  al) . 

9.  "Multivariant  Technique  for  Multi-Class  Pattern  Recognition,"  Appl  Opt, 
19,  1758-1761,  June  1980  (Hester  and  Casasent). 

10.  "Optical  Fourier  Transform  Techniques  for  Advanced  Fourier  Spectroscopy," 
Appl  Opt,  19,  2034-2037,  June  1980  (Casasent  and  Psaltis). 

11.  "Non-Linear  t-E  Curve  Effects  in  an  Optical  Correlator,"  Opt  Comm,  34 , 
4-6,  July  1980  (Kumar  and  Casasent) . 

12.  "Correlation  of  Images  with  Random  Contrast  Reversals,"  SPIE,  238,  156- 
165,  July  1980  (Barniv,  Mostafavi,  Casasent). 

13.  "A  Laser  Diode  Lensless  MSF-HOE  Correlator,"  Appl  Opt,  19,  2653-2654, 
August  1980  (Caimi  et  al). 


